Re: [openstack-dev] [all] Bringing back auto-abandon (was: Review metrics - what do we want to measure?)

2014-09-10 Thread Matthew Oliver
Hey all,

I have written a abandon notifier script for swift. The idea is it does a
gerrit query for changes that haven't been updated for 4 weeks, has a
negative score and isn't WIP. It then sends an email notification warning
the owner that the change is being considered as an abandoned change and
gives them some options on what they can do about it, and gives them a
deadline (2 weeks from when the notification was sent).

Once 2 weeks have passed since the notification was sent, and the change
still hasn't been touched, then it is added to a list of abandoned changes
(a simple html list [cause I'm lazy and so it can be easily loaded into
swift in future]). The change and notification metadata is all managed in a
MySQL database so statistical analysis can applied later.

The script doesn't log in to gerrit, but could be extended to login and
actually auto abandon, but haven't quite got that far. At the moment I can
bring the list up at the swift meeting and cores can go abandon things (or
they can check the report directly if they want).

This is new and only went live a few weeks ago, so still seeing how it
goes, but so far, in the database I see quite a few changes flagged as
deleted (which means they have been updated (assuming due to the
notification being sent).

All of the configuration, includiing the email template can be found in the
yaml config file.

This could be extended to look at other/all projects or could be run by
each project them selves. If anyone is interested the code is here:
https://github.com/matthewoliver/swift_abandon_notifier

And the report (abandoned changes 14 days without response from the initial
notification sent) can be found:
http://abandoner.oliver.net.au


On Thu, Sep 11, 2014 at 4:04 AM, Steven Hardy  wrote:

> On Wed, Sep 10, 2014 at 04:37:14PM +, Jeremy Stanley wrote:
> > On 2014-09-10 09:57:17 +0100 (+0100), Steven Hardy wrote:
> > > I can understand this argument, and perhaps an auto-abandon with a
> > > long period like say a month for the submitter to address comments
> > > and reset the clock would be a reasonable compromise?
> >
> > Perhaps, but the old script lacked a bunch of things we'd need for
> > this (it didn't understand/skip WIP changes, it also abandoned
> > changes with no reviews at all, it didn't support for the version of
> > Gerrit we're running now) and was tied to old cruft we've excised
> > (particularly the Launchpad->Gerrit group membership sync code) so
> > would likely need to be rewritten mostly from scratch. Probably if
> > we were to do it right, it would be something like a new kind of
> > periodic meta-job job in Zuul which could be selectively added to
> > projects who want it.
>
> Ok, so lets collect requirements for a smarter auto-abandon script, and
> figure out how feasable it is.
>
> > > The problem we have now, is there is a constantly expanding queue
> > > of zombie reviews, where the submitter got some negative feedback
> > > and, for whatever reason, has chosen not to address it.
> >
> > I agree that's a problem, but I'm unconvinced it's because we lack
> > some system to automatically make them go away.
>
> It is though, because the current system puts a never-ending increasing
> load on a finite and hard-to-grow resource (core reviewers).
>
> Clearly documented auto-abandon puts the load on the never-ending
> increasing group of new contributors.
>
> > > For example, in my "Incoming reviews" queue on the gerrit
> > > dashboard, I've got reviews I (or someone) -1'd over four months
> > > ago, with zero feedback from the submitter, what value is there to
> > > these reviews cluttering up the dashboard for every reviewer?
> >
> > The moment you, as a core reviewer, abandon one of those with a
> > friendly note it will cease to clutter up the dashboard for anyone.
> > But doing so also gives you an opportunity to possibly notice that
> > this is actually a valuable bug fix/enhancement for your project and
> > take it over. If it gets automatically abandoned because some random
> > reviewer did a drive-by -1 about whitespace preferences a month ago,
> > then you may never know.
>
> I think the issue is, in practice, this is not happening, for the following
> reasons:
>
> - There's no "friendly" way, as a core reviewer, of saying to a contributor
>   that their patch is abandoned.  I've never abandoned someone elses patch,
>   because I see it as carrying the same message as -2 reviews - it's a
>   strongly negative message to send to a contributor.
>
> - As a core reviewer, typically I have zero visibility of whether a
>   contributor is actually working on the patch, so what's a reasonable time
>   to wait before "taking over" or abandoning a patch?  Not all folks are
>   active on IRC and even if they are, it's more -core cycles spent tracking
>   folks down and chasing them (often repeatedly) for status updates.
>
> A clearly communicated patch-expiry system solves both of these issues.
>
> > > To make matters worse J

Re: [openstack-dev] [nova] policy on old / virtually abandoned patches

2014-12-05 Thread Matthew Oliver
I have a script that does 95% of what you want:

https://github.com/matthewoliver/swift_abandon_notifier

We are using it for swift reviews. At the moment the only thing it doesn't
do is actually abandon, it instead sends a warning email and waits n days
(2 weeks by default) for action, if it still turns up it adds it to a list
of abandoned changes.

Eg: http://abandoner.oliver.net.au

So anything that appears in that list can be abandoned by a core.

Feel free to use it (just uses a yaml file for configuration) and we can
all benefit from enhancements made ;)

Matt
On Dec 5, 2014 11:06 PM, "Daniel P. Berrange"  wrote:

> On Tue, Nov 18, 2014 at 07:06:59AM -0500, Sean Dague wrote:
> > Nova currently has 197 patches that have seen no activity in the last 4
> > weeks (project:openstack/nova age:4weeks status:open).
>
> On a somewhat related note, nova-specs currently has 17 specs
> open against specs/juno, most with -2 votes. I think we should
> just mass-abandon anything still touching the specs/juno directory.
> If people cared about them still they would have submitted for
> specs/kilo.
>
> So any objection to killing everything in the list below:
>
>
> +-+---+--+---+-+--+
> | URL | Subject
>| Created  | Tests | Reviews | Workflow |
>
> +-+---+--+---+-+--+
> | https://review.openstack.org/86938  | Add tasks to the v3 API
>  | 237 days |  1| -2  |  |
> | https://review.openstack.org/88334  | Add support for USB controller
> | 231 days |  1| -2  |  |
> | https://review.openstack.org/89766  | Add useful metrics into
> utilization based scheduli... | 226 days |  1| -2  |  |
> | https://review.openstack.org/90239  | Blueprint for Cinder Multi attach
> volumes | 224 days |  1| -2  |  |
> | https://review.openstack.org/90647  | Add utilization based weighers on
> top of MetricsWe... | 221 days |  1| -2  |  |
> | https://review.openstack.org/96543  | Smart Scheduler (Solver
> Scheduler) - Constraint ba... | 189 days |  1| -2  |  |
> | https://review.openstack.org/97441  | Add nova spec for
> bp/isnot-operator   | 185 days |  1| -2  |
> |
> | https://review.openstack.org/99476  | Dedicate aggregates for specific
> tenants  | 176 days |  1| -2  |  |
> | https://review.openstack.org/99576  | Add client token to CreateServer
> | 176 days |  1| -2  |  |
> | https://review.openstack.org/101921 | Spec for Neutron migration
> feature| 164 days |  1| -2  |  |
> | https://review.openstack.org/103617 | Support Identity V3 API
>  | 157 days |  1| -1  |  |
> | https://review.openstack.org/105385 | Leverage the features of IBM GPFS
> to store cached ... | 150 days |  1| -2  |  |
> | https://review.openstack.org/108582 | Add ironic boot mode filters
> | 136 days |  1| -2  |  |
> | https://review.openstack.org/110639 | Blueprint for the implementation
> of Nested Quota D... | 127 days |  1| -2  |  |
> | https://review.openstack.org/111308 | Added VirtProperties object
> blueprint | 125 days |  1| -2  |  |
> | https://review.openstack.org/111745 | Improve instance boot time
> | 122 days |  1| -2  |  |
> | https://review.openstack.org/116280 | Add a new filter to implement
> project isolation fe... | 104 days |  1| -2  |  |
>
> +-+---+--+---+-+--+
>
>
> Regards,
> Daniel
> --
> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/
> :|
> |: http://libvirt.org  -o- http://virt-manager.org
> :|
> |: http://autobuild.org   -o- http://search.cpan.org/~danberr/
> :|
> |: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc
> :|
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Heat] Meeting time redux

2014-04-23 Thread Matthew Oliver
Seeing as its a global project, there will never be a great time.
Alternating in nice, but that has the potential for someone missing
important events for having their say cause they may just wait to the next
more convenient meeting.

We could have meetings based off where most contributors are, but that has
the potential of stifling other contributors who want to get more involved
(like me, but luckily I'm in Oz so a good timezone for the current
meetings).

We can keep them as they are, or keep it democratic, I think that the PTL
should try and attend most meetings as they have been elected to help guide
the project. Seeing as they are elected and should be there how about
having a meeting time that best suits them? Tho that would effect me this
time :)

We vote for PTL so it is democratic, it's always going to be a bad time
somewhere in the world, so this makes it fair. And everyone who wants to be
involved will always know the time.
Best part is, if someone doesn't like it, let them run for PTL.

Anyway, that's my 2 cents on the matter.

Matt
On Apr 24, 2014 5:17 AM, "Zane Bitter"  wrote:

> At the beginning of this year we introduced alternating times for the Heat
> weekly IRC meeting, in the hope that our contributors in Asia would be able
> to join us. The consensus is that this hasn't worked out as well as we had
> hoped - even the new time falls at 8am in Beijing, so folks are regularly
> unable to make the meeting. It also falls at 5pm on the west coast of the
> US, so folks from there are also regularly unable to make the meeting too.
> And of course it is in the middle of the night for Europe, so the meeting
> room looks like a ghost town.
>
> Since we are in a new development cycle (with the PTL in a different
> location) and daylight savings has kicked in/out in many places, let's
> review our options. Here are our choices as I see it:
>
> * Keep going with the current system or some minor tweak to it.
>
> * Flip the alternate meeting by 12 hours to 1200 UTC. (8pm in China, late
> night in Oceania, early-morning on the east coast of the US and we lose the
> rest of the US.)
>
> * Lose all US-based folks and have a meeting for the rest of the world at
> around 0700 UTC. (US-based folks include me, so I would have to ask someone
> else to take care of passing on messages-from-the-PTL.)
>
> * Abandon the alternating meetings altogether.
>
> What would people prefer? I'd particularly like to hear from folks based
> in Asia what times would enable them to regularly attend, while still
> ensuring there are other people there to talk to ;)
>
> thanks,
> Zane.
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] How to re-compile Devstack Code

2014-04-23 Thread Matthew Oliver
If you put OFFLINE=True in your localrc (or local.conf) file, then you can:
./unstack.sh; ./stack.sh
without the stack.sh attempting to recheck out the latest code.

Matt
On Apr 24, 2014 3:49 PM, "shiva m"  wrote:

> Hi,
>
> I have Devstack havana setup on Ubuntu 13.10. I am trying to modify some
> files  in /opt/stack/* folder. How do I re-compile the Devstack to make my
> changes get effect.
>
> Does unstacking and stacking works?. I see unstacking and stacking
> installs everything fresh.
>
> Correct me  if  wrong.
>
> Thanks,
> Shiva
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [swift] Allow hostname for nodes in Ring

2014-10-14 Thread Matthew Oliver
Hey Hisashi ,

It does indeed look abandoned. So if you want to take it over sure that's
OK :)

1) It isn't a bug, so no bug report is needed. You could raise a BP if you
wanted, jump on the freenode #openstack-swift channel if you want to
discuss.

2) You can do one of two things:
  - Continue where this one left off, in which case pull down the change
from gerrit and start working on it. But if you do this make sure you add a
'Co-Authored-By: name ' to attribute the work that was
already done by the original author*. And start working on it in a new
change.

  - Start from scratch, but then you will be starting again.

Regards,
Matt

*
https://wiki.openstack.org/wiki/GitCommitMessages#Including_external_references
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Swift] Swift 2.0 release candidate

2014-06-22 Thread Matthew Oliver
Nice work swift devs!

Storage policies were a great idea and now they exist!

Awesome work!

Matt
On Jun 23, 2014 2:04 PM, "John Dickinson"  wrote:

> Through extensive work from the entirety of the Swift dev team over the
> past year, storage policies have landed in Swift. Last Friday, we merged
> commit 1feaf6e2 which brings storage polices into master.
>
> I especially would like to publicly thank Paul Luse (Intel), Clay Gerrard
> (SwiftStack), and Sam Merritt (SwiftStack) for providing such tremendous
> focus, dedication, awesome ideas, and leadership to getting this feature
> designed, written, and merged.
>
> For those that don't know, storage policies are a way to take the global
> footprint of your Swift cluster and choose what subset of hardware to store
> the data on and how to store it across that subset of hardware. For
> example, a single Swift cluster can now have data segmented by geographic
> region or performance tier. Additionally, each policy can have a different
> replication factor, which enables high replication for local access (e.g.
> one copy in every PoP) or low replication for some data (e.g. image
> thumbnails or transcoded video).
>
> Storage policies is the necessary building block to allow non-replicated
> storage (i.e. erasure codes) in Swift, a feature that we are continuing to
> develop.
>
> Full documentation, including design, usage, and upgrade notes, can be
> found at http://swift.openstack.org/overview_policies.html.
>
> With this commit landing, we have tagged Swift 2.0.0.rc1. We are now
> having a two week QA period to allow community members to play with it in
> their labs. At the end of the RC period, we'll formally release Swift 2.0.
> The current target for this is Thursday July 3 (although I realize that
> discovered issues and the US holiday may put this at risk).
>
> In addition to participating in the OpenStack integrated release cycle,
> Swift makes semantically-versioned releases throughout the year. Because of
> the scope of the storage policies changes and because you cannot safely
> downgrade your cluster after configuring a second policy (i.e. you'd lose
> access to that data if you go to a pre-storage-policies release), we have
> chosen to bump the major version number to 2.
>
> Note that deployers can still upgrade to this version with no client
> downtime and still safely downgrade until multiple policies are configured.
>
> The full CHANGELOG for the 2.0 release is at
> https://github.com/openstack/swift/blob/master/CHANGELOG.
>
> If you are using Swift, please read over the docs, download the tarball
> from http://tarballs.openstack.org/swift/swift-2.0.0.rc1.tar.gz, and let
> us know what you find.
>
> --John
>
>
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Infra][Solum][Mistral] New class of requirements for Stackforge projects

2014-06-25 Thread Matthew Oliver
On Jun 26, 2014 12:12 PM, "Angus Salkeld" 
wrote:
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 25/06/14 15:13, Clark Boylan wrote:
> > On Tue, Jun 24, 2014 at 9:54 PM, Adrian Otto 
wrote:
> >> Hello,
> >>
> >> Solum has run into a constraint with the current scheme for
requirements management within the OpenStack CI system. We have a proposal
for dealing with this constraint that involves making a contribution to
openstack-infra. This message explains the constraint, and our proposal for
addressing it.
> >>
> >> == Background ==
> >>
> >> OpenStack uses a list of global requirements in the requirements
repo[1], and each project has it’s own requirements.txt and
test-requirements.txt files. The requirements are satisfied by gate jobs
using pip configured to use the pypi.openstack.org mirror, which is
periodically updated with new content from pypi.python.org. One motivation
for doing this is that pypi.python.org may not be as fast or as reliable as
a local mirror. The gate/check jobs for the projects use the OpenStack
internal pypi mirror to ensure stability.
> >>
> >> The OpenStack CI system will sync up the requirements across all the
official projects and will create reviews in the participating projects for
any mis-matches. Solum is one of these projects, and enjoys this feature.
> >>
> >> Another motivation is so that users of OpenStack will have one single
set of python package requirements/dependencies to install and run the
individual OpenStack components.
> >>
> >> == Problem ==
> >>
> >> Stackforge projects listed in openstack/requirements/projects.txt that
decide to depend on each other (for example, Solum wanting to list
mistralclient as a requirement) are unable to, because they are not yet
integrated, and are not listed in
openstack/requirements/global-requirements.txt yet. This means that in
order to depend on each other, a project must withdraw from projects.txt
and begin using pip with pypi.poython.org to satisfy all of their
requirements.I strongly dislike this option.
> >>
> >> Mistral is still evolving rapidly, and we don’t think it makes sense
for them to pursue integration wight now. The upstream distributions who
include packages to support OpenStack will also prefer not to deal with a
requirement that will be cutting a new version every week or two in order
to satisfy evolving needs as Solum and other consumers of Mistral help
refine how it works.
> >>
> >> == Proposal ==
> >>
> >> We want the best of both worlds. We want the freedom to innovate and
use new software for a limited selection of stackforge projects, and still
use the OpenStack pypi server to satisfy my regular requirements. We want
the speed and reliability of using our local mirror, and users of Solum to
use a matching set of requirements for all the things that we use, and
integrated projects use. We want to continue getting the reviews that bring
us up to date with new requirements versions.
> >>
> >> We propose that we submit an enhancement to the gate/check job setup
that will:
> >>
> >> 1) Begin (as it does today) by satisfying global-requirements.txt and
my local project’s requirements.txt and test-requirements.txt using the
local OpenStack pypi mirror.
> >> 2) After all requirements are satisfied, check the name of my project.
If it begins with ‘stackforge/‘ then look for a stackforge-requirements.txt
file. If one exists, reconfigure pip to switch to use pypi.python.org, and
satisfy the requirements listed in the file. We will list mistralclient
there, and get the latest tagged/released version of that.
> >>
> > I am reasonably sure that if you remove yourself from the
> > openstack/requirements project list this is basically how it will
> > work. Pip is configured to use the OpenStack mirror and fall back on
> > pypi.python.org for packages not available on the OpenStack mirror
> > [2]. So I don't think there is any work to do here with additional
> > requirements files. It should just work. Adding a new requirements
> > file will just make things more confusing for packagers and consumers
> > of your software.
>
> Adrian I know this is not the optimal solution, but I think this is
> the most pragmatic solution (esp. given we need to progress and not be
held
> up by this), most stackforge projects are in the same boat as us.
> As far as pypi breakages (most are easily fixable by restricting the
> package versions if we get an issue with a new release
> of *random-api-breaking-package*).
>

I've looked through the infra choose mirror code, and Clark is correct. If
the project isn't in the projects.txt file they will only access to
pypi.openstack.org however if removed then it will first check
pypi.openstack.org and then fall back to to pypi.python.org. I think the
only real solution is what Angus mentioned, remove yourself from
projects.txt at least until all your dependencies can be provided by
pypi.openstack.org or another solution is put into place. In the mean time
you can at least progress and

Re: [openstack-dev] [ALL] Removing the tox==1.6.1 pin

2014-07-25 Thread Matthew Oliver
Just tested tox 1.7.2 with swift on my Dev box and tox -epy27 runs fine.

So seems Swift isn't affected by this.

Matt
 On Jul 26, 2014 7:44 AM, "Clark Boylan"  wrote:

> Hello,
>
> The recent release of tox 1.7.2 has fixed the {posargs} interpolation
> issues we had with newer tox which forced us to be pinned to tox==1.6.1.
> Before we can remove the pin and start telling people to use latest tox
> we need to address a new default behavior in tox.
>
> New tox sets a random PYTHONHASHSEED value by default. Arguably this is
> a good thing as it forces you to write code that handles unknown hash
> seeds, but unfortunately many projects' unittests don't currently deal
> with this very well. A work around is to hard set a PYTHONHASHSEED of 0
> in tox.ini files. I have begun to propose these changes to the projects
> that I have tested and found to not handle random seeds. It would be
> great if we could get these reviewed and merged so that infra can update
> the version of tox used on our side.
>
> I probably won't be able to test every single project and propose fixes
> with backports to stable branches for everything. It would be a massive
> help if individual projects tested and proposed fixes as necessary too
> (these changes will need to be backported to stable branches). You can
> test by running `tox -epy27` in your project with tox version 1.7.2. If
> that fails add PYTHONHASHSEED=0 as in
> https://review.openstack.org/#/c/109700/ and rerun `tox -epy27` to
> confirm that succeeds.
>
> This will get us over the immediate hump of the tox upgrade, but we
> should also start work to make our tests run with random hashes. This
> shouldn't be too hard to do as it will be a self gating change once
> infra is able to update the version of tox used in the gate. Most of the
> issues appear related to dict entry ordering. I have gone ahead and
> created https://bugs.launchpad.net/cinder/+bug/1348818 to track this
> work.
>
> Thank you,
> Clark
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Swift] Meeting time change

2015-05-26 Thread Matthew Oliver
+1 to what Kota said!

Matt
On May 27, 2015 11:39 AM, "Kota TSUYUZAKI" 
wrote:

>  Thanks John and all swifters for taking care of Asian Pasific and
> Australian time zone. Much appreciated :)
>
> Kota
>
> (2015/05/26 4:32), John Dickinson wrote:
>
> Based on discussion over at the summit and over the last few weeks, the Swift 
> team meeting time has changed.
>
> The new meeting time is 2100UTC on Wednesdays in #openstack-meeting.
>
> The meeting agenda is tracked at 
> https://wiki.openstack.org/wiki/Meetings/Swift
>
>
> --John
>
>
>
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [swift] zones and partition

2016-02-21 Thread Matthew Oliver
Kiru,

That just means you have put even weight on all your drives, so your
telling swift to store it that way.

So short answer is  there is more to it then that. Sure evenly balanced
makes life easier. But it doesn't have to be the case. You can set drive
weights and overload factor to tune/balance data placement throughout the
cluster. Further you have more then just regions and zones, swift knows
about servers and disks. And will always attempt to keep the objects and
disburse and durable as possible.

If there is ever a case for a some partitions to have 2 replicas on the one
zone, then you'd find they live on different servers or if there is only 1
server, different disks. Therefore adding more failure domains, the better
your data is durability stored.

Regards,
Matt

On Mon, Feb 22, 2016 at 2:00 PM, Kirubakaran Kaliannan <
kiru...@zadarastorage.com> wrote:

>
>
> Hi,
>
>
>
> I have 3 ZONEs, with different capacity in each. Say I have 4 X 1TB disk
>  (r0z1 - 1TB, r0z2 - 1TB,r0 z3 - 2TB ).
>
>
>
> The ring builder (rebalance code), keep ¼-partitions of all 3 replica in
> Zone-3. This is the current default  behavior from the rebalance code.
>
> This puts pressure to the storage user to evenly increase the storage
> capacity across the zones. Is this is the correct understanding I have ?
>
>
>
> If so, Why have we chosen this approach, rather cant we enforce zone based
> partition (but the partition size on Z1 and Z2 may be lesser than Z3) ?
>
> This makes sure we have 100% zone level protection and not loss of data on
> 1 zone failure ?
>
>
>
> Thanks,
>
> -kiru
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] Can we stop global requirements update?

2017-04-19 Thread Matthew Oliver
We have started this work. I've been working on:
https://review.openstack.org/#/c/444718/

Which will do requirement checks, as specified in the Pike PTG ehterpad for
Tuesday morning:
https://etherpad.openstack.org/p/relmgt-stable-requirements-ptg-pike (line
40+).

Once done, Tony and I were going to start testing it on the experimental
pipeline for Swift and Nova.

Regards,
Matt

On Thu, Apr 20, 2017 at 2:34 AM, Doug Hellmann 
wrote:

> Excerpts from Clark Boylan's message of 2017-04-19 08:10:43 -0700:
> > On Wed, Apr 19, 2017, at 05:54 AM, Julien Danjou wrote:
> > > Hoy,
> > >
> > > So Gnocchi gate is all broken (agan) because it depends on "pbr"
> and
> > > some new release of oslo.* depends on pbr!=2.1.0.
> > >
> > > Neither Gnocchi nor Oslo cares about whatever bug there is in pbr 2.1.0
> > > that got in banished by requirements Gods. It does not prevent it to be
> > > used e.g. to install the software or get version information. But it
> > > does break anything that is not in OpenStack because well, pip installs
> > > the latest pbr (2.1.0) and then oslo.* is unhappy about it.
> >
> > It actually breaks everything, including OpenStack. Shade and others are
> > affected by this as well. The specific problem here is that PBR is a
> > setup_requires which means it gets installed by easy_install before
> > anything else. This means that the requirements restrictions are not
> > applied to it (neither are the constraints). So you get latest PBR from
> > easy_install then later when something checks the requirements
> > (pkg_resources console script entrypoints?) they break because latest
> > PBR isn't allowed.
> >
> > We need to stop pinning PBR and more generally stop pinning any
> > setup_requires (there are a few more now since setuptools itself is
> > starting to use that to list its deps rather than bundling them).
> >
> > > So I understand the culprit is probably pip installation scheme, and we
> > > can blame him until we fix it. I'm also trying to push pbr 2.2.0 to
> > > avoid the entire issue.
> >
> > Yes, a new release of PBR undoing the "pin" is the current sane step
> > forward for fixing this particular issue. Monty also suggested that we
> > gate global requirements changes on requiring changes not pin any
> > setup_requires.
> >
> > > But for the future, could we stop updating the requirements in oslo
> libs
> > > for no good reason? just because some random OpenStack project hit a
> bug
> > > somewhere?
> > >
> > > For example, I've removed requirements update on tooz¹ for more than a
> > > year now, which did not break *anything* in the meantime, proving that
> > > this process is giving more problem than solutions. Oslo libs doing
> that
> > > automatic update introduce more pain for all consumers than anything
> (at
> > > least not in OpenStack).
> >
> > You are likely largely shielded by the constraints list here which is
> > derivative of the global requirements list. Basically by using
> > constraints you get distilled global requirements and even without being
> > part of the requirements updates you'd be shielded from breakages when
> > installed via something like devstack or other deployment method using
> > constraints.
> >
> > > So if we care about Oslo users outside OpenStack, I beg us to stop this
> > > crazyness. If we don't, we'll just spend time getting rid of Oslo over
> > > the long term…
> >
> > I think we know from experience that just stopping (eg reverting to the
> > situation we had before requirements and constraints) would lead to
> > sadness. Installations would frequently be impossible due to some
> > unresolvable error in dependency resolution. Do you have some
> > alternative in mind? Perhaps we loosen the in project requirements and
> > explicitly state that constraints are known to work due to testing and
> > users should use constraints? That would give users control to manage
> > their own constraints list too if they wish. Maybe we do this in
> > libraries while continuing to be more specific in applications?
>
> At the meeting in Austin, the requirements team accepted my proposal
> to stop syncing requirements updates into projects, as described
> in https://etherpad.openstack.org/p/ocata-requirements-notes
>
> We haven't been able to find anyone to work on the implementation,
> though. I is my understanding that Tony did contact the Telemetry
> and Swift teams, who are most interested in this area of change,
> about devoting some resources to the tasks outlined in the proposal.
>
> Doug
>
> >
> > >
> > > My 2c,
> > >
> > > Cheers,
> > >
> > > ¹ Unless some API changed in a dep and we needed to raise the dep,
> > > obviously.
> > >
> > > --
> > > Julien Danjou
> > > # Free Software hacker
> > > # https://julien.danjou.info
> >
> > I don't have all the answers, but am fairly certain the situation we
> > have today is better than the one from several years ago. It is just not
> > perfect. I think we are better served by refining the current setup or
> 

Re: [openstack-dev] [storlets][mascot] mascot for the storlets project

2017-04-24 Thread Matthew Oliver
Looks cool, I'd stick it on my laptop :)

On Mon, Apr 24, 2017 at 4:43 PM,  wrote:

> +1
>
> Looks so nice to me!
>
> Thanks for the great illustration, Heidi!
>
>
>
>
>
> Thank you,
>
> Takashi
>
>
>
> *From:* Eran Rom [mailto:e...@itsonlyme.name]
> *Sent:* Monday, April 24, 2017 2:35 AM
> *To:* OpenStack Development Mailing List (not for usage questions) <
> openstack-dev@lists.openstack.org>
> *Subject:* [openstack-dev] [storlets][mascot] mascot for the storlets
> project
>
>
>
> Hi All,
>
> Any feedbacks?
>
> Personally I like it.
>
>
>
> Thanks,
>
> Eran
>
>
>
> Begin forwarded message:
>
>
>
> *From: *Heidi Joy Tretheway 
>
> *Subject: Re: mascot for the storlets project*
>
> *Date: *April 22, 2017 at 1:14:27 AM GMT+3
>
> *To: *Eran Rom 
>
>
>
> Hi Eran,
>
>
>
> I took your feedback on the storklet to our illustrators and they came up
> with this revised version. Would you please let me know what your team
> thinks?
>
>
>
> On Feb 28, 2017, at 11:52 AM, Heidi Joy Tretheway 
> wrote:
>
>
>
> Hi Eran,
>
> Thank you so much! These are great pictures to give me an idea of what a
> stork let should look like. I’ll share both with our illustration team.
>
> Best,
> Heidi Joy
>
>
> On Feb 28, 2017, at 11:43 AM, Eran Rom  wrote:
>
> Heidi Hi,
> In the PTG our team has chose ’storklet’ as a mascot.
> I was not aware until recently that there is such a word, and that it is
> the stork chick.
>
> Here is a couple of nice stoklet pictures I have found:
> http://www.arkive.org/marabou-stork/leptoptilos-
> crumeniferus/image-G69355.html (A marabou storklet )
> http://www.daufuskieislandconservancy.org/index.php?page=wood-stork (A
> wood stork storklet)
>
> Thanks very much!
> Eran
>
>
>
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [First Contact] [OUI] Call for Project Liaisons

2017-12-11 Thread Matthew Oliver
I'll put my hand up for my timezone. Especially as interest in my side of
the world grows I'd love to make sure there is someone available.

Project: Swift
Name: Matthew Oliver
IRC: mattolverau
email: m...@oliver.net.au
timezone: UTC +11 (AEDT)

Matt

On Tue, Dec 12, 2017 at 8:25 AM, Kendall Nelson 
wrote:

> Hello,
>
> As some of you may know there is a newly formed SIG focused on the initial
> interactions of someone new to the community called the First Contact SIG.
> While the team we have looking out for these first introductions is growing
> they can only get us so far. After we get them up to speed with the tools
> and community it is important that they are connected with someone who can
> get them up to speed on a project. While these SIG members cover a variety
> of projects, there are several projects that don’t have someone to help
> these newcomers.
>
> If you are interested in being a project liaison please reply to this
> thread with your name, irc nic, email, and timezone (UTC +/- x).
>
> Previously, Upstream Institute had started to collect a list of people
> interested in helping get new community members up to speed in the past and
> I think it would be good to unite these efforts. That list covers only 6
> projects: Cinder, Keystone, Nova, Neutron, QA, and Trove[1]. If you signed
> up before and are interested in this renewed effort please let me know!
>
> Even if everyone from the initial effort participates, there are still
> many projects missing from the list. How best to ensure the growth and
> success of your project than to invest in the people interested in
> contributing to your project?
>
> Thanks!
>
> Kendall Nelson (diablo_rojo)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [First Contact] [OUI] Call for Project Liaisons

2017-12-12 Thread Matthew Oliver
Initially I meant project liaison, but who am I to say no to helping new
people. I was new once, and had some great OpenStack mentors help me along
the way :)

Matt

On Tue, Dec 12, 2017 at 9:40 AM, Amy Marrich  wrote:

> Pretty sure he said both!:)
>
> Amy (spotz)
>
> On Mon, Dec 11, 2017 at 4:33 PM, Kendall Nelson 
> wrote:
>
>> Thank you Matt!
>>
>> To clarify, you want to be Project Liaison for sSwift? Or a member of the
>> SIG? Or both? :)
>>
>> -Kendall (diablo_rojo)
>>
>>
>> On Mon, Dec 11, 2017 at 2:18 PM Matthew Oliver 
>> wrote:
>>
>>> I'll put my hand up for my timezone. Especially as interest in my side
>>> of the world grows I'd love to make sure there is someone available.
>>>
>>> Project: Swift
>>> Name: Matthew Oliver
>>> IRC: mattolverau
>>> email: m...@oliver.net.au
>>> timezone: UTC +11 (AEDT)
>>>
>>> Matt
>>>
>>> On Tue, Dec 12, 2017 at 8:25 AM, Kendall Nelson 
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> As some of you may know there is a newly formed SIG focused on the
>>>> initial interactions of someone new to the community called the First
>>>> Contact SIG. While the team we have looking out for these first
>>>> introductions is growing they can only get us so far. After we get them up
>>>> to speed with the tools and community it is important that they are
>>>> connected with someone who can get them up to speed on a project. While
>>>> these SIG members cover a variety of projects, there are several projects
>>>> that don’t have someone to help these newcomers.
>>>>
>>>> If you are interested in being a project liaison please reply to this
>>>> thread with your name, irc nic, email, and timezone (UTC +/- x).
>>>>
>>>> Previously, Upstream Institute had started to collect a list of people
>>>> interested in helping get new community members up to speed in the past and
>>>> I think it would be good to unite these efforts. That list covers only 6
>>>> projects: Cinder, Keystone, Nova, Neutron, QA, and Trove[1]. If you signed
>>>> up before and are interested in this renewed effort please let me know!
>>>>
>>>> Even if everyone from the initial effort participates, there are still
>>>> many projects missing from the list. How best to ensure the growth and
>>>> success of your project than to invest in the people interested in
>>>> contributing to your project?
>>>>
>>>> Thanks!
>>>>
>>>> Kendall Nelson (diablo_rojo)
>>>>
>>>> 
>>>> __
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe: openstack-dev-requ...@lists.op
>>>> enstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>> 
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.op
>>> enstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [First Contact] [SIG] Presence at the PTG

2017-12-14 Thread Matthew Oliver
Sounds great.. Still haven't got a green light from my employer whether
they'd send me or not yet tho.

But if I can get to the PTG I'll be there :)

Matt

On Fri, Dec 15, 2017 at 8:41 AM, Kendall Nelson 
wrote:

> Yeah that should be doable :)
>
> As we get a little closer, I can draft an agenda and add OUI to it.
>
> -Kendall
>
> On Thu, Dec 14, 2017 at 1:37 PM Jay Bryant 
> wrote:
>
>> Sounds like a good plan.  Hopefully we can do some OUI work face to face
>> as well?
>>
>> Jay
>>
>> On Thu, Dec 14, 2017, 2:55 PM Kendall Nelson 
>> wrote:
>>
>>> Hello,
>>>
>>> It came up in a discussion today that it might be good to get together
>>> and discuss all the activities around on boarding and various other initial
>>> interactions to get us all on the same page and a little more
>>> organized/established.
>>>
>>> Given that SIGs have space the beginning of the week (Mon/Tuesday), I am
>>> proposing that we meet for one of these days. If you are interested, please
>>> let me know so I can get a headcount.
>>>
>>> -Kendall (diablo_rojo)
>>>
>> 
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
>>> unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
>> unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [requirements][FFE] oslosphinx

2016-09-04 Thread Matthew Oliver
Thanks Tony,

Yup that's my understanding too.

Like you said, thanks to the upper-constraints having the correct version
(4.7.0), and the fact that currently the swiftclient has oslosphinx
unconstrained, means that the online documentation builds OK and the docs
job passes in the gate. Further, anyone trying to build the
python-swiftclient documentation from a fresh install will also be fine, as
pip should download the latest.

We definitely don't want to rock the boat while everyone is trying to
stabilise for the next release, so I think option 2 or 3 are two good
solutions. Personally, because of upper constraints and the fact that there
is a rather simple (or 2 [upgrade or comment out a line]) work arounds, I'm
personally leading for option 2. Rather then adding something hacky that
will be removed in month or so.

I'm also happy to go with option 2 and volunteer to raise option 3 as a
patch if it does turn out to be a major problem or others.

Anyway that's my 2 cents,

Matt

On Mon, Sep 5, 2016 at 1:56 PM, Tony Breeds  wrote:

> On Sat, Sep 03, 2016 at 05:12:27PM +0200, Ondrej Novy wrote:
> > Hi,
> >
> > https://review.openstack.org/365253
> >
> > released newton version of python-swiftclient 3.1.0 requires newer
> version
> > of oslosphinx. Without it, docs can't be built.
> >
> > So i prepared fix:
> > https://review.openstack.org/#/c/365239/1
> >
> > But gating says: no, you should update global requirements first. So I'm
> > doing it.
> >
> > Please consider granting an exception. Thanks.
>
> I think we need a little more detail.
>
> As I understand it the issue is that projects which set
> 'show_other_versions' in
> html_theme_options will fail to build if it gets oslosphinx < 4.7.0.  A
> quick
> look at [1] shows that this is only python-swiftclient.
>
> More generally however the docs *are* building because we always get
> oslosphinx 4.7.0 [2] So this review is about correctness rather than
> fixing a
> breakage.
>
> Please correct any of the above if I have it wrong.
>
> I'll outline our options on the assumption that I'm reasonably close in my
> problem analysis.
>
> From my POV we have a few options:
>
> 1) Grant the FFE
>Here are some stats about oslosphinx:
> Package: oslosphinx [oslosphinx!=3.4.0,>=2.5.0] (used by 223
> projects)
>Taking review 365253 would force a minor semver bump a release of 76
>projects + and deeper dependencies in libraries.  Server/service
> projects
>will get the requirements bump in RC1.  This has a pretty major impact
> on
>the release team at a time they're trying to stabilise things.
>
> 2) Fix it on master when the freeze is lifted
>As I said the docs are building for python-swiftclient, so we could
>acknowledge that we have a problem and fix it for master.  Newton will
> always
>be broken but I think we've missed the boat on raising the minimum here
> due
>to the scale of the impact.  We can't fix it on stable/newton as that's
>against the stable-policy.
>
> 3) Add a hack to python-swiftclient work around the problem.
>This is a terrible and I admit it's papering over an issue with out
> tools/process but we could do something like:
> ---
> diff --git a/doc/source/conf.py b/doc/source/conf.py
> index 5af77b2..f72b8d5 100644
> --- a/doc/source/conf.py
> +++ b/doc/source/conf.py
> @@ -111,7 +111,14 @@ pygments_style = 'sphinx'
>  # documentation.
>  # html_theme_options = {}
>
> -html_theme_options = {'show_other_versions': True}
> +from pkg_resources import get_distribution, parse_version
> +
> +oslosphinx_installed = parse_version(get_distribution('oslosphinx').
> version)
> +
> +if oslosphinx_installed >= parse_version('4.7.0'):
> +html_theme_options = {'show_other_versions': True}
> +else:
> +html_theme_options = {}
> ---
>
> My opinion is that option 2 is the least worst, taking option 3 will be up
> to
> the swift and oslo teams.
>
>
> Yours Tony.
>
> [1] http://codesearch.openstack.org/?q=show_other_versions&i=
> nope&files=&repos=
> [2] Because:
> 1) The projects supports constraints and gets 4.7.0 ; or
> 2) The projects is unconstrained and pip picks the latest (4.7.0)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [ptg] Dublin PTG schedule up

2018-02-01 Thread Matthew Oliver
Sweet thanks Thierry,

Only issue is I see what days things are happening, but not what rooms
things are in. Unless I'm failing at reading a table.

Matt

On Fri, Feb 2, 2018 at 8:02 AM, Thierry Carrez 
wrote:

> Hi everyone,
>
> The schedule for the Dublin PTG is now posted on the PTG website:
> https://www.openstack.org/ptg#tab_schedule
>
> I'll post on this thread if anything changes, but it's pretty unlikely
> at this point.
>
> Note that we have a lot of available rooms on Monday/Tuesday to discuss
> additional topics. If you think of something we should really take half
> a day to discuss, please add it to the following etherpad:
>
> https://etherpad.openstack.org/p/PTG-Dublin-missing-topics
>
> If there is consensus it's a good topic and we agree on a time where to
> fit it, we could add it to the schedule.
>
> For smalled things (like 90-min discussions) we can book time
> dynamically during the event thanks to the new PTGbot features.
>
> See you there !
>
> --
> Thierry Carrez (ttx)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [ptg] Dublin PTG schedule up

2018-02-02 Thread Matthew Oliver
Ahh makes sense. Thanks Thierry, I can tell this isn't your first Rodeo :)

Matt

On Fri, Feb 2, 2018 at 7:52 PM, Thierry Carrez 
wrote:

> Matthew Oliver wrote:
> > Sweet thanks Thierry,
> >
> > Only issue is I see what days things are happening, but not what rooms
> > things are in. Unless I'm failing at reading a table.
>
> Yes, that's by design. We publish the days early so that people can
> fine-tune their travel plans and start organizing. We are still
> finalizing the exact room assignments, though. Those will be ready by
> the time the event starts :)
>
> --
> Thierry
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [First Contact][SIG] Weekly Meeting

2018-03-15 Thread Matthew Oliver
sorry I missed the meeting, I've been off with family related gastro fun,
fun times... first me and then the pregnant wife which means spending the
whole morning yesterday in hostpital re-hydrating the wife, just as a
precaution to keep the baby safe. good news was the toddler wasn't hit as
bad as us.

Anyway, I'll be at the next one. My apologies, this week ended up being
kind of a write off for me :(

Matt

On Wed, Mar 14, 2018 at 10:13 AM, Kendall Nelson 
wrote:

> Hello!
>
> [1] has been merged and we have an agenda [2] so we are full steam ahead
> for the upcoming meeting!
>
> Our inaugural First Contact SIG meeting will be in #openstack-meeting at
> 0800 UTC Wednesday!
>
> Hope to see you all in ~9 hours!
>
> -Kendall (diablo_rojo)
>
> [1]https://review.openstack.org/#/c/549849/
> [2] https://wiki.openstack.org/wiki/First_Contact_SIG#Meeting_Agenda
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [swift] limit/marker/count behavior when limit not given

2016-12-18 Thread Matthew Oliver
On Mon, Dec 19, 2016 at 11:54 AM, Monty Taylor  wrote:

> Hey all,
>
> In the docs for listing containers:
>
> http://developer.openstack.org/api-ref/object-storage/?
> expanded=show-account-details-and-list-containers-detail
>
> it says that limit is optional. However, if you give it
> full_listing=True, swiftclient always sends at least a second query with
> a marker set to the name of the last element of the original return
> value. This leads me believe there is a default value for limit server
> side that swiftclient is trying to account for.
>
> Looking through capabilities, it seems there is:
>
> swift.account_listing_limit
> swift.container_listing_limit
>
>
Yeah there is a server side limit, and as you surmise you can find it when
you query the cluster for the information (swift capabilities or  curl -i
http:///info |python -m json.tool).

The swift client will look at capabilities and be smart about it.

We limit the size of the response to the server size limit. full_listing is
swiftclient trying to be helpful and paginate for you.


> I'm guessing swift.account_listing_limit would apply to getting the list
> of containers, and container_listing_limit would apply to listing
> objects in a container, yeah?
>

Exactly, yes.


>
> Relatedly, there is a X-Account-Container-Count header. Testing against
> rackspace public shows me that it contains the total count of containers
> even if I provide a limit parameter:
>
> >>> r=c.get('/?format=json&limit=2')
> >>> r.headers['X-Account-Container-Count']
> '4'
>
> Would that also be the case if I had hit the
> swift.account_listing_limit limit?


Yup, the container-count (account metadata) and object-count (container
metadata) are how many are in the account/container NOT in the response. As
it's gathering the metadata information from the account/container. If you
fire off a HEAD (or stat in swiftclient) you'll get the same counts.




> I ask because it seems like it would be ever-so-mildly more efficient to
> look at the returned count and the total count and only do the loop if
> there is a mismatch - but I'm curious if there is something I should
> account for I'm not seeing. (I'm working on making direct REST calls,
> but starting with swiftclient's behavior to make sure I'm doing things
> right)
>
> Also - the question about limit behavior when not given seems like it
> could be added to the docs - and I'd be happy to make that patch if I'm
> reading things right.
>

Looking at the API docs briefly, this could be more clearly stated.
Especially as the first paragraph in the Account GET mentions if you get
'limit' back then there are more objects, and the first paragraph of the
Container GET mentions there being a default limit on the server side if
the parameters are omitted:
http://developer.openstack.org/api-ref/object-storage/?expanded=show-account-details-and-list-containers-detail,show-container-details-and-list-objects-detail

So yeah could definitely be cleaned up.


> Thanks!
> Monty
>
> No thank you :)

> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


I was talking to Richard from horizon while they've been building a new
Swift UI, and for displaying many objects, they've mentioned liking someway
of knowing if there are any more results, so they can continue to paginate
as people scroll etc, our discussions have gone further then just knowing
if there's more now. And so I plan on having a play with helping them
paginate better on the Swift side for Horizon.

At the moment, if you get limit results back (where limit is what you
define or defaults to the server default) then there is a good chance there
are more objects. Just keeping track of the total can be a little
problematic as It's possible for that count to grow or shrink between
requests. So if you hit limit, then there is probably more and do another
request is easiest.

Regards,
Matt
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [First Contact] [SIG] [PTL] Project Liaisons - Please update list

2018-08-14 Thread Matthew Oliver
Greetings new and continuing PTLs!

Now that the PTL elections are over may I ask those of you who haven't done
so already to head on over to the First Contact SIG Project Liaison list
[0] and make sure you details, especially timezone has been filled out?

We of the First Contact SIG are striving to provide a place for new
contributors to come and get help, advice, and to better get connected to
our awesome online community. But we can only to that if we have coverage
of projects. As Kendall mentioned in a previous email [1], unless the
liaison for a project is filled by a volunteer, or better yet multiple
volunteers, the liaison for a project it will default to the PTL.

Please update your details in the list and provide your timezone, this will
not only let us know you've looked, but also shows us that you are engaged.
What do liaisons do? You'll be the point of call for new contributors. This
could be a SIG member adding you to a gerrit review of a new contributor to
help engage and show some review love, or being introduced to new
contributors to your project via email or IRC.

Why is your timezone important? Because new contributors are coming from
all around the globe and ideally we'd love to have not only project
coverage but timezone as well. This way be can get support and
encouragement to new contributors even in timezones that aren't covered.
For example, I'm in Australia, so I'm available in the APAC timezone. I can
reach out and help a new contributor even if they're interested in a
project I'm not familiar with, we are all one team, and their problems may
be generic enough I can help. If not, I can point them to a liaison that
can, hopefully in a closer timezone, but if not at least get them connected
via email.

As a SIG we are trying to help support new contributors, get them engaged,
and lower the bar to entry. Especially keeping there excitement up when
they get their first few patches in. Please partner with us and lets grow
the community.

Regards,
Matt

[0] - https://wiki.openstack.org/wiki/First_Contact_SIG#Project_Liaisons
[1] -
http://lists.openstack.org/pipermail/openstack-dev/2018-June/131264.html
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][cinder][glance][osc][sdk] Image Encryption for OpenStack (proposal)

2018-10-15 Thread Matthew Oliver
Just an FYI, it doesn't solved cached images, but Swift does support at
rest encryption, so if using the Swift store backend you can at least know
your image on disk on the storage nodes would be safe.
We still need to add more functionality like key rotation, but we do
integrate with kmip sevices or barbican.

Still could be a good idea for other projects. I wasn't the one who wrote
the Swift at-rest encryption but happy to, probably badly, help answer
questions cause we might have some interesting lessons learned.

Matt

On Tue, Oct 16, 2018 at 12:36 AM Josephine Seifert <
josephine.seif...@secustack.com> wrote:

> Hello OpenStack developers,
>
> we have made an etherpad as there were a few questions concerning
> the library we want to use for the encryption and decryption method:
>
>
> https://etherpad.openstack.org/p/library-for-image-encryption-and-decryption
>
>
> Am 11.10.2018 um 15:10 schrieb Josephine Seifert:
> > Am 08.10.2018 um 17:16 schrieb Markus Hentsch:
> >> Dear OpenStack developers,
> >>
> >> as you suggested, we have written individual specs for Nova [1] and
> >> Cinder [2] so far and will write another spec for Glance soon. We'd
> >> appreciate any feedback and reviews on the specs :)
> >>
> >> Thank you in advance,
> >> Markus Hentsch
> >>
> >> [1] https://review.openstack.org/#/c/608696/
> >> [2] https://review.openstack.org/#/c/608663/
> >>
> >>
> >>
> __
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > The spec for Glance is also on gerrit now:
> >
> > https://review.openstack.org/#/c/609667/
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev