[openstack-dev] [Horizon] Some thoughts about Horizon's test suite

[Richard Jones]

I'm relatively new to Horizon, but I come with a bunch of experience with
Python more generally. I've contributed a couple of small patches to
Horizon in an effort to get more familiar with the codebase. Julie Pichon's
blog post about testing in Horizon has been invaluable <
http://www.jpichon.net/blog/2013/07/testing-in-horizon/>.

Very recently I attempted to fix a simple bug in which a Panel was being
displayed when it shouldn't have been. The resultant 5-line fix ended up
breaking 498 of the 1048 unit tests in the suite. I estimated that it would
take about a week's effort to address all the failing tests. For more
information see <
http://mechanicalcat.net/richard/log/Python/When_testing_goes_bad>

In talking about the issues I've hit I found quite a few people both in
#openstack-horizon and other places who are quite forthright in their
desire to get rid of all use of mox. I heartily endorse this motion.
Indeed, I would be willing to start the process of weeding mox out of the
suite entirely, one test module at a time, if there is support for such an
effort from Horizon core/TL.

Removing mox is one (admittedly very large) piece of the puzzle though. As
I mentioned in my blog post, there are other structural issues that would
need to be addressed also; the use of unittest.TestCase and the varying
methods of mocking.

At the moment, making a fix like the one I attempted is prohibitively
expensive to do, and thus won't be done because Horizon's test suite has
become too restrictive to change (unnecessarily so). I'd like to see that
change.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Some thoughts about Horizon's test suite

[Richard Jones]

Thanks for your thoughts Radomir. The nova api in question is memoized so
it'll only be called once per request. Caching it for longer would be a
very good idea, but that then brings into play deeper knowledge than I have
about how long to cache things like nova extension configuration. Also, I
looked into this see whether we could use a nicer existing memoizing system
(one with a timeout, that doesn't use weakref and will clean out stale
entries), but none of them will handle the existence of the varying request
parameter, so more work would be required to build our own solution. It's
still something I'd like to see done.

But that's not really the point of this email, as you note :)



On 29 August 2014 19:46, Radomir Dopieralski  wrote:

> On 29/08/14 04:22, Richard Jones wrote:
>
>
> > Very recently I attempted to fix a simple bug in which a Panel was being
> > displayed when it shouldn't have been. The resultant 5-line fix ended up
> > breaking 498 of the 1048 unit tests in the suite. I estimated that it
> > would take about a week's effort to address all the failing tests. For
> > more information see
> > <http://mechanicalcat.net/richard/log/Python/When_testing_goes_bad>
>
> Having read that, I can't help but comment that maybe, just maybe,
> making an API call on each an every request to Horizon is not such a
> great idea after all, and should be very well thought out, as it is
> costly. In particular, it should be investigated if the call could be
> made only on some requests? That would have the side effect of breaking
> much fewer tests.
>
> But I agree that mox is horrible in that it effectively freezes the
> implementation details of the tested unit, instead of testing its behavior.
>
> --
> Radomir Dopieralski
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Supporting Javascript clients calling OpenStack APIs

[Richard Jones]

[This is Horizon-related but affects every service in OpenStack, hence no
filter in the subject]

I would like for OpenStack to support browser-based Javascript API clients.
Currently this is not possible because of cross-origin resource blocking in
Javascript clients - that is, given some Javascript hosted on
"https://horizon.company.com/"; you cannot, for example, call from that
Javascript code to an API on "https://apis.company.com:5000/v2.0/tokens"; to
authenticate with Keystone.

There are three solutions to this problem:

1. the Horizon solution, in which those APIs are proxied by a very thick
   layer of additional Python API, plus some Python view code with some
   Javascript on the top only calling the Horizon view code,
2. add CORS support to all the OpenStack APIs though a new WSGI middleware
   (for example oslo.middleware.cors) and configured into each of the API
   services individually since they all exist on different "origin"
   host:port combinations, or
3. a new web service that proxies all the APIs and serves the static
   Javascript (etc) content from the one origin (host). APIs are then served
   from new URL roots "/name/" where the name is from the serviceCatalog
   entry. Static content can be served from "/static/". The serviceCatalog
from
   keystone will be rewritten on the fly to point the API publicURLs at the
   new service. Requests are no longer cross-origin.

I have implemented options 2 and 3 as an exercise to see how horrid each one
is.


== CORS Middleware ==

For those wanting a bit of background, I have written up a spec for oslo
that
talks about how this could work: https://review.openstack.org/#/c/119485/

The middleware option results in a reasonably nice bit of middleware. It's
short and relatively easy to test. The big problem with it comes in
configuring it in all the APIs. The configuration for the middleware takes
two forms:

1. hooking oslo.middleware.cors into the WSGI pipeline (there's more than
   one in each API),
2. adding the CORS configuration itself for the middleware in the API's main
   configuration file (eg. keystone.conf or nova.conf).

So for each service, that's two configuration files *and* the kicker is that
the paste configuration file is non-trivially different in almost every
case.

That's a lot of work, and confusing for deployers. Configuration management
tools can ease *some* of this burden (the *.conf files) but those paste
files
are a bit of a mess :(

Once the config change is in place, it works (well, except for an issue I
ran
into relating to oslo.middleware.sizelimit which I'll go into in another
place).

The implementation hasn't been pushed up for review as I'm not sure it
should
be. I can do this if people wish me to.


== New Single-Point API Service ==

Actually, this is not horrid in any way - unless that publicURL rewriting
gives you the heebie-jeebies.

It works, and offers us some nice additional features like being able to
host
the service behind SSL without needing to get a bazillion certificates. And
maybe load balancing. And maybe API access filtering.

I note that https://openrepose.org already exists to be *something* like
this, but it's not *precisely* what I'm proposing. Also Java euwww ;)


So, I propose that the idea of CORS-in-all-the-things as an idea be
put aside as unworkable.

I intend to pursue the single-point API service that I have described as a
way of moving forward in prototyping a pure-Javascript OpenStack Dashboard.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Supporting Javascript clients calling OpenStack APIs

[Richard Jones]

On 12 September 2014 07:50, Adam Young  wrote:

>  On 09/11/2014 03:15 AM, Richard Jones wrote:
>
>  [This is Horizon-related but affects every service in OpenStack, hence no
> filter in the subject]
>
>  I would like for OpenStack to support browser-based Javascript API
> clients.
> Currently this is not possible because of cross-origin resource blocking in
> Javascript clients - that is, given some Javascript hosted on
> "https://horizon.company.com/"; you cannot, for example, call from that
> Javascript code to an API on "https://apis.company.com:5000/v2.0/tokens";
> to
> authenticate with Keystone.
>
>  There are three solutions to this problem:
>
>  1. the Horizon solution, in which those APIs are proxied by a very thick
>layer of additional Python API, plus some Python view code with some
>Javascript on the top only calling the Horizon view code,
> 2. add CORS support to all the OpenStack APIs though a new WSGI middleware
>(for example oslo.middleware.cors) and configured into each of the API
>services individually since they all exist on different "origin"
>host:port combinations, or
> 3. a new web service that proxies all the APIs and serves the static
>Javascript (etc) content from the one origin (host). APIs are then
> served
>from new URL roots "/name/" where the name is from the serviceCatalog
>entry. Static content can be served from "/static/". The serviceCatalog
> from
>keystone will be rewritten on the fly to point the API publicURLs at the
>new service. Requests are no longer cross-origin.
>
>  I have implemented options 2 and 3 as an exercise to see how horrid each
> one
>  is.
>
>
> I don't think these are mutually exclusive.  I can see people wanting
> either in some deployments.
>

I think I agree :)


== CORS Middleware ==
>
>  For those wanting a bit of background, I have written up a spec for oslo
> that
> talks about how this could work: https://review.openstack.org/#/c/119485/
>
>  The middleware option results in a reasonably nice bit of middleware.
> It's
> short and relatively easy to test. The big problem with it comes in
> configuring it in all the APIs. The configuration for the middleware takes
> two forms:
>
>  1. hooking oslo.middleware.cors into the WSGI pipeline (there's more than
>one in each API),
> 2. adding the CORS configuration itself for the middleware in the API's
> main
>configuration file (eg. keystone.conf or nova.conf).
>
>  So for each service, that's two configuration files *and* the kicker is
> that
> the paste configuration file is non-trivially different in almost every
> case.
>
> This is one reason I thought that it should be done by auth_token
> middleware.  The other reason is that I don't think we want to blanket
> accept CORS from everywhere, but instead we should do so based on the
> service catalog.
>

It's very important to understand that CORS is entirely advisory. Nothing
is required to pay any attention to it. Modern browsers do, of course, but
in the absence of a browser initiating a CORS conversation (by sending an
Origin header) the CORS middleware should do nothing whatsoever. A GET
request will still return the body of data requested - it's just that the
browser will see the CORS header and block an application from seeing that
data. Security through access controls, XSS protections, etc. must be
handled by other mechanisms.


For a POC deployment, for a small company, all-in-one,  what you are doing
> shouild be fine, but then, if you were running all of your services that
> way, in one web server, you wouldn't need CORS either.
>

This isn't possible in the current OpenStack model - all the APIs run on
different ports, which count as different origins for cross-origin resource
issues (origin is defined as {protocol, host, port}).


So, lets have these two approaches work in parallel.  THe proxy will get
> things goint while we work out the CORS approach.
>

I will look at submitting my middleware for inclusion anyway then.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Supporting Javascript clients calling OpenStack APIs

[Richard Jones]

On 11 September 2014 18:00, Robert Collins 
wrote:

> FWIW I'm very much in favour of having a single host API - I was
> looking at doing that in Apache for TripleO deployments anyway, due to
> the better SSL deployment characteristics. We then would register the
> actual single host endpoint in publicURL.
>

Yep, basically.



> How would that work for multiple regions with javascript - can you
> switch hosts fairly easily ?
>

I'm not confident I'm completely across the implications of regions, but I
believe there's nothing about them that makes anything planned here break.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Supporting Javascript clients calling OpenStack APIs

[Richard Jones]

On 12 September 2014 09:24, Richard Jones  wrote:

> On 12 September 2014 07:50, Adam Young  wrote:
>
>> So, lets have these two approaches work in parallel.  THe proxy will get
>> things goint while we work out the CORS approach.
>>
>
> I will look at submitting my middleware for inclusion anyway then.
>

Submitted to oslo.middleware https://review.openstack.org/#/c/120964/


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon]Dev environment default login

[Richard Jones]

I needs to point at a devstack or openstack installation for login to work.

On 12 September 2014 15:50, Rajdeep Dua  wrote:

> Hi,
> I have setup a local dev environment with a custom dashboard using
> instructions below
>
> http://docs.openstack.org/developer/horizon/topics/tutorial.html
>
> Horizon started using ./run_tests.sh --runserver 0.0.0.0:8877
> What is the default password for admin login?
>
> Note : it is not pointing to any openstack installation yet
>
> Thanks
> Rajdeep
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] OpenStack bootstrapping hour - Friday Sept 19th - 3pm EST

[Richard Jones]

This is a great idea, thanks!

On 16 September 2014 08:56, Sean Dague  wrote:

> A few of us have decided to pull together a regular (cadence to be
> determined) video series taking on deep dives inside of OpenStack,
> looking at code, explaining why things work that way, and fielding
> questions from anyone interested.
>
> For lack of a better title, I've declared it OpenStack Bootstrapping Hour.
>
> Episode 0 - Mock best practices will kick off this Friday, Sept 19th,
> from 3pm - 4pm EST. Our experts for this will be Jay Pipes and Dan
> Smith. It will be done as a Google Hangout on Air, which means there
> will be a live youtube stream while it's on, and a recorded youtube
> video that's publicly accessible once we're done.
>
> We'll be using an etherpad during the broadcast to provide links to the
> content people are looking at, as well as capture questions. That will
> be our backchannel, and audience participation forum, with the advantage
> that it creates a nice concise document at the end of the broadcast that
> pairs well with the video. (Also: the tech test showed that while code
> examples are perfectly viewable during in the final video, during the
> live stream they are a little hard to read, etherpad links will help
> people follow along at home).
>
> Assuming this turns out to be useful, we're thinking about lots of other
> deep dives. The intent is that these are indepth dives. We as a
> community have learned so many things over the last 4 years, but as
> OpenStack has gotten so large, being familiar with more than a narrow
> slice is hard. This is hopefully a part of the solution to address that.
> As I've told others, if nothing else, I'm looking forward to learning a
> ton in the process.
>
> Final links for the hangout + etherpad will be posted a little later in
> the week. Mostly wanted to make people aware it was coming.
>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone][Horizon] CORS and Federation

[Richard Jones]

CORS for all of OpenStack is possible once the oslo middleware lands*, but
as you note it's only one of many elements to be considered when exposing
the APIs to browsers. There is no current support for CSRF protection in
the OpenStack APIs, for example. I believe that sort of functionality
belongs in an intermediary between the APIs and the browser.


Richard

* https://review.openstack.org/#/c/120964/

On 17 September 2014 08:59, Gabriel Hurley 
wrote:

> This is generally the right plan. The hard parts are in getting people to
> deploy it correctly and securely, and handling fallback cases for lack of
> browser support, etc.
>
> What we really don't want to do is to encourage people to set
> "Access-Control-Allow-Origin: *" type headers or other such nonsense simply
> because it's too much work to do things correctly. This becomes especially
> challenging for federated clouds.
>
> I would encourage looking at the problem of adding all the necessary
> headers for CORS as an OpenStack-wide issue. Once you figure it out for
> Keystone, the next logical step is to want to make calls from the browser
> directly to all the other service endpoints, and each service is going to
> have to respond with the correct CORS headers
> ("Access-Control-Allow-Methods" and "Access-Control-Allow-Headers" are
> particularly fun ones for projects like Glance or Swift). A common
> middleware and means of configuring it will go a long way to easing user
> pain and spurring adoption of the new mechanisms. It will help the Horizon
> team substantially in the long run to do it consistently and predictably
> across the stack.
>
> As a side-note, once we're in the realm of handling all this sensitive
> data with the browser as a middleman, encouraging people to configure
> things like CSP is probably also a good idea to make sure we're not loading
> malicious scripts or other resources.
>
> Securing a browser-centric world is a tricky realm... let's make sure we
> get it right. :-)
>
>  - Gabriel
>
> > -Original Message-
> > From: Adam Young [mailto:ayo...@redhat.com]
> > Sent: Tuesday, September 16, 2014 3:40 PM
> > To: OpenStack Development Mailing List
> > Subject: [openstack-dev] [Keystone][Horizon] CORS and Federation
> >
> > Phase one for dealing with Federation can be done with CORS support
> solely
> > for Keystone/Horizon  integration:
> >
> > 1.  Horizon Login page creates Javascript to do AJAX call to Keystone 2.
> > Keystone generates a token 3.  Javascript reads token out of response and
> > sends it to Horizon.
> >
> > This should support Kerberos, X509, and Password auth;  the Keystone team
> > is discussing how to advertise mechanisms, lets leave the onus on us to
> solve
> > that one and get back in a timely manner.
> >
> > For Federation, the handshake is a little more complex, and there might
> be a
> > need for some sort of popup window for the user to log in to their home
> > SAML provider.  Its several more AJAX calls, but the end effect should
> be the
> > same:  get a standard Keystone token and hand it to Horizon.
> >
> > This would mean that Horizon would have to validate tokens the same way
> > as any other endpoint.  That should not be too hard, but there is a
> little bit of
> > "create a user, get a token, make a call" logic that currently lives
> only in
> > keystonemiddleware/auth_token;  Its a solvable problem.
> >
> > This approach will support the straight Javascript approach that Richard
> Jones
> > discussed;  Keystone behind a proxy will work this way without CORS
> > support.  If CORS  can be sorted out for the other services, we can do
> straight
> > Javascript without the Proxy.  I see it as phased approach with this
> being the
> > first phase.
> >
> >
> >
> >
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone][Horizon] CORS and Federation

[Richard Jones]

You're quite probably correct - going through the OWASP threat list in more
detail is on my TODO. That was just off the top of my head as something
that has me concerned but I've not investigated it thoroughly.

On 17 September 2014 14:15, Adam Young  wrote:

>  On 09/16/2014 08:56 PM, Richard Jones wrote:
>
> CORS for all of OpenStack is possible once the oslo middleware lands*, but
> as you note it's only one of many elements to be considered when exposing
> the APIs to browsers. There is no current support for CSRF protection in
> the OpenStack APIs, for example. I believe that sort of functionality
> belongs in an intermediary between the APIs and the browser.
>
>
> Typically, CSRF is done by writing a customer header.  Why wouldn't the
> -X-Auth-Token header qualify?  Its not a cookie, automatically added.  So,
> CORS support would be necesary for horizon to send the token on a request
> to Nova, but no other site would be able to do that.  No?
>
>
>
>
>  Richard
>
>  * https://review.openstack.org/#/c/120964/
>
> On 17 September 2014 08:59, Gabriel Hurley 
> wrote:
>
>> This is generally the right plan. The hard parts are in getting people to
>> deploy it correctly and securely, and handling fallback cases for lack of
>> browser support, etc.
>>
>> What we really don't want to do is to encourage people to set
>> "Access-Control-Allow-Origin: *" type headers or other such nonsense simply
>> because it's too much work to do things correctly. This becomes especially
>> challenging for federated clouds.
>>
>> I would encourage looking at the problem of adding all the necessary
>> headers for CORS as an OpenStack-wide issue. Once you figure it out for
>> Keystone, the next logical step is to want to make calls from the browser
>> directly to all the other service endpoints, and each service is going to
>> have to respond with the correct CORS headers
>> ("Access-Control-Allow-Methods" and "Access-Control-Allow-Headers" are
>> particularly fun ones for projects like Glance or Swift). A common
>> middleware and means of configuring it will go a long way to easing user
>> pain and spurring adoption of the new mechanisms. It will help the Horizon
>> team substantially in the long run to do it consistently and predictably
>> across the stack.
>>
>> As a side-note, once we're in the realm of handling all this sensitive
>> data with the browser as a middleman, encouraging people to configure
>> things like CSP is probably also a good idea to make sure we're not loading
>> malicious scripts or other resources.
>>
>> Securing a browser-centric world is a tricky realm... let's make sure we
>> get it right. :-)
>>
>>  - Gabriel
>>
>> > -Original Message-
>> > From: Adam Young [mailto:ayo...@redhat.com]
>> > Sent: Tuesday, September 16, 2014 3:40 PM
>> > To: OpenStack Development Mailing List
>> > Subject: [openstack-dev] [Keystone][Horizon] CORS and Federation
>> >
>> > Phase one for dealing with Federation can be done with CORS support
>> solely
>> > for Keystone/Horizon  integration:
>> >
>> > 1.  Horizon Login page creates Javascript to do AJAX call to Keystone 2.
>> > Keystone generates a token 3.  Javascript reads token out of response
>> and
>> > sends it to Horizon.
>> >
>> > This should support Kerberos, X509, and Password auth;  the Keystone
>> team
>> > is discussing how to advertise mechanisms, lets leave the onus on us to
>> solve
>> > that one and get back in a timely manner.
>> >
>> > For Federation, the handshake is a little more complex, and there might
>> be a
>> > need for some sort of popup window for the user to log in to their home
>> > SAML provider.  Its several more AJAX calls, but the end effect should
>> be the
>> > same:  get a standard Keystone token and hand it to Horizon.
>> >
>> > This would mean that Horizon would have to validate tokens the same way
>> > as any other endpoint.  That should not be too hard, but there is a
>> little bit of
>> > "create a user, get a token, make a call" logic that currently lives
>> only in
>> > keystonemiddleware/auth_token;  Its a solvable problem.
>> >
>> > This approach will support the straight Javascript approach that
>> Richard Jones
>> > discussed;  Keystone behind a proxy will work this way without CORS
>> > support.  If CORS  can be sorted out for the other service

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 19 November 2014 03:14, Radomir Dopieralski 
wrote:

> On 18/11/14 12:54, Matthias Runge wrote:
> > On Tue, Nov 18, 2014 at 09:36:00PM +1100, Richard Jones wrote:
> >> I guess I got the message that turning bower packages into system
> packages
> >> was something that the Linux packagers were not keen on. Did I get the
> >> message wrong there? If so, *and* we can get the bower stuff through
> #infra
> >> and global-requirements then yes, we should totally try to avoid adding
> the
> >> xstatic layer :)
> >
> > For me, it's more work to turn packages into bower, or to translate
> > bower packages to system packages.
> >
> > But that is purely because we don't have bower packaged currently in
> Fedora.
> > I would vote for the cleaner way (whatever that is).
> >
> > XStatic is obviously not the cleanest way, but a good compromise in most
> > situations.
>
> The way I thought about it, we would simply replace the Bower packages
> with the corresponding system packages, by just changing the path in the
> settings.py file.
>
> Maybe an example would help illustrate it.
> Currently we have something like this:
>
> STATICFILES_DIRS = [
> ('horizon/lib/angular',
>xstatic.main.XStatic(xstatic.pkg.angular).base_dir),
> ('horizon/lib/angular',
>xstatic.main.XStatic(xstatic.pkg.angular_cookies).base_dir),
> ('horizon/lib/angular',
>xstatic.main.XStatic(xstatic.pkg.angular_mock).base_dir),
> ('horizon/lib/bootstrap_datepicker',
>xstatic.main.XStatic(xstatic.pkg.bootstrap_datepicker).base_dir),
> ('bootstrap',
>xstatic.main.XStatic(xstatic.pkg.bootstrap_scss).base_dir),
> ...
> ]
>
> We would replace that with:
>
> STATICFILES_DIRS = [
> ('horizon/lib/angular',
>os.path.join(BASE_DIR, 'bower_modules/angular'),
> ...
> ]
>
> or wherever bower puts the files. Now, the packagers would replace those
> lines with:
>
> STATICFILES_DIRS = [
> ('horizon/lib/angular',
>'/usr/lib/javascript/angular'),
> ...
> ]
>
> or whatever the packages in their distribution use. It's less work than
> having to make a whole xstatic package just to put that single path in
> there. The horizon system package would already depend on all the
> javascript library packages, so we are sure the files are there.
>  <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>

+1 to all that, except I'd recommend using django-bower to handle the
static collection stuff. It's not documented but django-bower has a setting
BOWER_COMPONENTS_ROOT which would make the above transition much simpler.
You leave it alone for local dev, and packagers setup BOWER_COMPONENTS_ROOT
to '/usr/lib/javascript/' or wherever.

bower also has a concept of entry points - there a "main" value in the
bower.json which identifies the Javascript, CSS, font and other files to
include in the index.html to have the library loaded into your application.
Saves a bunch of manual editing to get things right when you include the
library. Sadly, the current django-bower plugin doesn't use any of that,
though it does handle the collectstatic stuff.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] proposal: alternating weekly meeting time [doodle poll created]

[Richard Jones]

Thanks everyone who has responded. I'm going to leave the poll open until
the weekend to allow for stragglers to get their times in, and then close
it and we can see what the results are.

The poll is at https://doodle.com/47h3f35nad62ncnf scroll to the far right
to set your timezone.


 Richard

On 12 November 2014 12:45, Richard Jones  wrote:

> I have set up a doodle poll to let folk enter their preferred times. It's
> in UTC/GMT (/London time, because doodle) so use something like
> http://everytimezone.com/ to figure that out :)
>
> https://doodle.com/47h3f35nad62ncnf
>
>
>  Richard
>
> On 11 November 2014 18:46, Matthias Runge  wrote:
>
>> On 11/11/14 08:09, Richard Jones wrote:
>> > Hi all,
>> >
>> > At the summit meetup last week I proposed that the Horizon weekly
>> > meeting time alternate between the current time and something more
>> > suitable for those of us closer to UTC+10. I'd like to get an indication
>> > of the interest in this, and I'll look into getting a second meeting
>> > time booked for alternating weeks based on your feedback.
>> >
>> > As a starting point, I'd like to suggest the times alternate between UTC
>> >  and 1600 (the current time).
>>
>> Sadly, both times don't work for me. I would propose something like 8
>> UTC, which should work for most folks located in EU and east, or 18 UTC.
>>
>> Matthias
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

I've just had a long discussion with #infra folk about the
global-requirements thing, which deviated (quite naturally) into a
discussion about packaging (and their thoughts were in line with where
Radomir and I are heading).

In their view, bower components don't need to be in global-requirements:

- there are no other projects that use bower components, so we don't need
to ensure cross-project compatibility
- we can vet new versions of bower components as part of standard Horizon
change review


Richard


On 19 November 2014 06:43, Richard Jones  wrote:

> On 19 November 2014 03:14, Radomir Dopieralski 
> wrote:
>
>> On 18/11/14 12:54, Matthias Runge wrote:
>> > On Tue, Nov 18, 2014 at 09:36:00PM +1100, Richard Jones wrote:
>> >> I guess I got the message that turning bower packages into system
>> packages
>> >> was something that the Linux packagers were not keen on. Did I get the
>> >> message wrong there? If so, *and* we can get the bower stuff through
>> #infra
>> >> and global-requirements then yes, we should totally try to avoid
>> adding the
>> >> xstatic layer :)
>> >
>> > For me, it's more work to turn packages into bower, or to translate
>> > bower packages to system packages.
>> >
>> > But that is purely because we don't have bower packaged currently in
>> Fedora.
>> > I would vote for the cleaner way (whatever that is).
>> >
>> > XStatic is obviously not the cleanest way, but a good compromise in most
>> > situations.
>>
>> The way I thought about it, we would simply replace the Bower packages
>> with the corresponding system packages, by just changing the path in the
>> settings.py file.
>>
>> Maybe an example would help illustrate it.
>> Currently we have something like this:
>>
>> STATICFILES_DIRS = [
>> ('horizon/lib/angular',
>>xstatic.main.XStatic(xstatic.pkg.angular).base_dir),
>> ('horizon/lib/angular',
>>xstatic.main.XStatic(xstatic.pkg.angular_cookies).base_dir),
>> ('horizon/lib/angular',
>>xstatic.main.XStatic(xstatic.pkg.angular_mock).base_dir),
>> ('horizon/lib/bootstrap_datepicker',
>>xstatic.main.XStatic(xstatic.pkg.bootstrap_datepicker).base_dir),
>> ('bootstrap',
>>xstatic.main.XStatic(xstatic.pkg.bootstrap_scss).base_dir),
>> ...
>> ]
>>
>> We would replace that with:
>>
>> STATICFILES_DIRS = [
>> ('horizon/lib/angular',
>>os.path.join(BASE_DIR, 'bower_modules/angular'),
>> ...
>> ]
>>
>> or wherever bower puts the files. Now, the packagers would replace those
>> lines with:
>>
>> STATICFILES_DIRS = [
>> ('horizon/lib/angular',
>>'/usr/lib/javascript/angular'),
>> ...
>> ]
>>
>> or whatever the packages in their distribution use. It's less work than
>> having to make a whole xstatic package just to put that single path in
>> there. The horizon system package would already depend on all the
>> javascript library packages, so we are sure the files are there.
>>  <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>>
>
> +1 to all that, except I'd recommend using django-bower to handle the
> static collection stuff. It's not documented but django-bower has a setting
> BOWER_COMPONENTS_ROOT which would make the above transition much simpler.
> You leave it alone for local dev, and packagers setup BOWER_COMPONENTS_ROOT
> to '/usr/lib/javascript/' or wherever.
>
> bower also has a concept of entry points - there a "main" value in the
> bower.json which identifies the Javascript, CSS, font and other files to
> include in the index.html to have the library loaded into your application.
> Saves a bunch of manual editing to get things right when you include the
> library. Sadly, the current django-bower plugin doesn't use any of that,
> though it does handle the collectstatic stuff.
>
>
>  Richard
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On Fri Nov 21 2014 at 4:06:51 AM Thomas Goirand  wrote:

> On 11/17/2014 06:54 PM, Radomir Dopieralski wrote:
> > - A tool, probably a script, that would help packaging the Bower
> > packages into DEB/RPM packages. I suspect the Debian/Fedora packagers
> > already have a semi-automatic solution for that.
>
> Nop. Bower isn't even packaged in Debian. Though I may try to do it
> (when I'm done with other Mirantis stuff like packaging Fuel for
> Debian...).
>

Just to be clear, it's not bower itself (the command-line tool) that needs
packaging, just the components that bower itself packages.



> On 11/18/2014 07:59 AM, Richard Jones wrote:
> > I was envisaging us creating a tool which generates xstatic packages
> > from bower packages. I'm not the first to think along these
> > lines
> http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html
>
> I think that's a very good idea!
>
> > I wrote the tool today, and you can find it here:
> >
> > https://github.com/r1chardj0n3s/flaming-shame
>
> AWESOME ! :)
> Then now, everyone is happy. Thank you.
>

Well, no, but at least it exists ;)



> On 11/18/2014 04:22 PM, Radomir Dopieralski wrote:
> > If we use Bower, we don't need to use Xstatic. It would be pure
> > overhead. Bower already takes care of tracking releases and versions,
> > and of bundling the files. All we need is a simple line in the
> > settings.py telling Django where it puts all the files -- we don't
> > really need Xstatic just for that. The packagers can then take those
> > Bower packages and turn them into system packages, and just add/change
> > the paths in settings.py to where they put the files. All in one
> > place.
>
> The issue is that there's often not just a single path, but a full
> directory structure to address. That is easily managed with a Debian
> xstatic package, not sure how it would be with Bower.
>

I'm not sure what the difference is (unless it's just related to the
Debian-specific historical issue you raise below). xstatic and bower are
remarkably similar a directory to be packaged and some meta-data describing
it.



> On 11/18/2014 06:36 PM, Richard Jones wrote:
> > I guess I got the message that turning bower packages into system
> > packages was something that the Linux packagers were not keen on.
>
> What I'm not a fan of, is that we'll have external dependencies being
> bumped all the time, with unexpected consequences. At least, with
> xstatic packages, we control what's going on (though I understand the
> overhead work problem).
>

The dependencies won't be bumped any faster than reviewers allow, though I
realise that's not necessarily going to make you sleep any easier. Hmm.



> By the way, I went into bower.io as I wanted to have a look. How do I
> download a binary package for let's say jasmin? When searching, it
> just links to github...
>

Again; bower is not npm! Jasmine is a command-line program which is
packaged by npm. Bower packages bundles of stuff that are included in web
applications. bower itself, a command-line tool, is packaged by npm, but
itself manages other packages which are not command-line things, but just
bundles of css, javascript, images, fonts, etc. that are resources for web
applications to use.



> On 11/19/2014 12:14 AM, Radomir Dopieralski wrote:
> > We would replace that with:
> >
> > STATICFILES_DIRS = [
> > ('horizon/lib/angular',
> >os.path.join(BASE_DIR, 'bower_modules/angular'),
> > ...
> > ]
>
> This would only work if upstream package directory structure is the same
> as the one in the distribution. For historical reason, that's
> unfortunately often not the case (sometimes because we want to keep
> backward compatibility in the distro because of reverse dependencies),
> and just changing the path wont make it work.
>
> On 11/19/2014 03:43 AM, Richard Jones wrote:
> > +1 to all that, except I'd recommend using django-bower to handle the
> > static collection stuff. It's not documented but django-bower has a
> > setting BOWER_COMPONENTS_ROOT which would make the above transition much
> > simpler. You leave it alone for local dev, and packagers setup
> > BOWER_COMPONENTS_ROOT to '/usr/lib/javascript/' or wherever.
>
> s/lib/share/
>
> However, I'm almost sure that wont be enough to make it work. For
> example, in Debian, we have /usr/share/javascript/angular.js, not just
> /usr/share/javascript/angular. So django-bower would be searching on the
> wrong path.
>

That is unfortunate. It may be that Debian therefore has to special-case
angular to h

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 21 November 2014 16:12, Thomas Goirand  wrote:

> On 11/21/2014 10:52 AM, Richard Jones wrote:
> > On 11/18/2014 04:22 PM, Radomir Dopieralski wrote:
> > > If we use Bower, we don't need to use Xstatic. It would be pure
> > > overhead. Bower already takes care of tracking releases and
> versions,
> > > and of bundling the files. All we need is a simple line in the
> > > settings.py telling Django where it puts all the files -- we don't
> > > really need Xstatic just for that. The packagers can then take
> those
> > > Bower packages and turn them into system packages, and just
> add/change
> > > the paths in settings.py to where they put the files. All in one
> > > place.
> >
> > The issue is that there's often not just a single path, but a full
> > directory structure to address. That is easily managed with a Debian
> > xstatic package, not sure how it would be with Bower.
> >
> >
> > I'm not sure what the difference is (unless it's just related to the
> > Debian-specific historical issue you raise below). xstatic and bower are
> > remarkably similar a directory to be packaged and some meta-data
> > describing it.
>
> Let me explain again then.
>
> Let's say there's python-xstatic-foo, and libjs-foo in Debian. If the
> directory structure of libjs-foo is very different from xstatic-foo, I
> can address that issue with symlinks within the xstatic package. Just
> changing the BASE_DIR may not be enough, as libjs-foo may have files
> organized in a very different way than in the upstream package for foo.
>

OK, so python-xstatic-foo can depend on libjs-foo and just symlink, fair
enough. I'm still not sure what makes bower unique in this respect,
although it'd be nice to avoid creating additional packages just to symlink
things around for bower, which I think is what you're getting at.


> By the way, I went into bower.io <http://bower.io> as I wanted to
> > have a look. How do I
> > download a binary package for let's say jasmin? When searching, it
> > just links to github...
> >
> >
> > Again; bower is not npm! Jasmine is a command-line program which is
> > packaged by npm. Bower packages bundles of stuff that are included in
> > web applications. bower itself, a command-line tool, is packaged by npm,
> > but itself manages other packages which are not command-line things, but
> > just bundles of css, javascript, images, fonts, etc. that are resources
> > for web applications to use.
>
> Sure. But how do I download a bower package then?
>

I'm not sure I understand the meaning behind this question. "bower install
angular" downloads a bower package called "angular".


> This would only work if upstream package directory structure is the
> same
> > as the one in the distribution. For historical reason, that's
> > unfortunately often not the case (sometimes because we want to keep
> > backward compatibility in the distro because of reverse
> dependencies),
> > and just changing the path wont make it work.
> >
> > On 11/19/2014 03:43 AM, Richard Jones wrote:
> > > +1 to all that, except I'd recommend using django-bower to handle
> the
> > > static collection stuff. It's not documented but django-bower has a
> > > setting BOWER_COMPONENTS_ROOT which would make the above
> > transition much
> > > simpler. You leave it alone for local dev, and packagers setup
> > > BOWER_COMPONENTS_ROOT to '/usr/lib/javascript/' or wherever.
> >
> > s/lib/share/
> >
> > However, I'm almost sure that wont be enough to make it work. For
> > example, in Debian, we have /usr/share/javascript/angular.__js, not
> just
> > /usr/share/javascript/angular. So django-bower would be searching on
> the
> > wrong path.
> >
> >
> > That is unfortunate. It may be that Debian therefore has to special-case
> > angular to handle that case.
>
> I wasn't making a point about Angular here. It's a general issue we have
> to take care of addressing correctly.
>
> > I think the general idea of following the component pattern set by bower
> > (separate directories with no risk of conflicts, and using the
> > bower.json meta-data to allow automatic configuration of the component)
> > is too good to dismiss though. Far less work for everyone, including
> > packagers.
> >
> > Perhaps the new packages should have "bower&qu

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

All the bower components I've used have included full source alongside any
minified versions.

On Sat, 22 Nov 2014 07:40 Fox, Kevin M  wrote:

> Simply having a git repository does not imply that its source.
>
> In fact, if its considered compiled (minified), I'm thinking the debian
> rules would prevent sourcing from it?
>
> Thanks,
> Kevin
> 
> From: Donald Stufft [don...@stufft.io]
> Sent: Friday, November 21, 2014 8:39 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [Horizon] the future of angularjs
> development  in Horizon
>
> > On Nov 21, 2014, at 11:32 AM, Jeremy Stanley  wrote:
> >
> > On 2014-11-21 07:31:36 -0500 (-0500), Donald Stufft wrote:
> >> You can’t. Bower doesn’t have “traditional” packages where you take a
> >> directory and archive it using tar/zip/whatever and then upload it to
> >> some repo. Bower has a registry which maps names to git URLs and then
> >> the bower CLI looks up that mapping, fetches the git repository and then
> >> uses that as the input to the “look at metadata and do stuff with files”
> >> part of the package manager instead of the output of an un-unarchival
> >> command.
> >
> > This raises interesting free software philosophy/license
> > questions... how do I redistribute (or even examine) the "source" of
> > a bower-managed package? Is there a way without actually
> > reverse-engineering the toolchain?
>
> Well it’s a git repository, so you could just clone it and look at it.
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] proposal: alternating weekly meeting time [doodle poll created]

[Richard Jones]

Thanks everyone, I've closed the poll. I'm sorry to say that there's no
combination of two timeslots which allows everyone to attend a meeting. Of
the 25 respondents, the best we can do is cater for 24 of you.

Optimising for the maximum number of attendees, the potential meeting times
are 2000 UTC Tuesday and 1000 UTC on one of Monday, Wednesday or Friday. In
all three cases the only person who has indicated they cannot attend is
Lifeless.

Unfortunately, David has indicated that he can't be present at the Tuesday
2000 UTC slot. Optimising for him as a required attendee for both meetings
means we lose an additional attendee, and gives us the Wednesday 2000 UTC
slot and a few options:

- Monday, Wednesday and Thursday at 1200 UTC (Lifeless and ygbo miss)
- Friday at 1200 UTC (Lifeless and Jaromir Coufal miss)

If anyone else would like to play with the timeslots and numbers, I can
pass on the excel sheet and my code :)

According to the meetings wiki page, we should be able to get an IRC room
at any of the above times.


  Richard

On Wed Nov 19 2014 at 9:15:40 AM Richard Jones 
wrote:

> Thanks everyone who has responded. I'm going to leave the poll open until
> the weekend to allow for stragglers to get their times in, and then close
> it and we can see what the results are.
>
> The poll is at https://doodle.com/47h3f35nad62ncnf scroll to the far
> right to set your timezone.
>
>
>  Richard
> On 12 November 2014 12:45, Richard Jones  wrote:
>
>> I have set up a doodle poll to let folk enter their preferred times. It's
>> in UTC/GMT (/London time, because doodle) so use something like
>> http://everytimezone.com/ to figure that out :)
>>
>> https://doodle.com/47h3f35nad62ncnf
>>
>>
>>  Richard
>>
>> On 11 November 2014 18:46, Matthias Runge  wrote:
>>
>>> On 11/11/14 08:09, Richard Jones wrote:
>>> > Hi all,
>>> >
>>> > At the summit meetup last week I proposed that the Horizon weekly
>>> > meeting time alternate between the current time and something more
>>> > suitable for those of us closer to UTC+10. I'd like to get an
>>> indication
>>> > of the interest in this, and I'll look into getting a second meeting
>>> > time booked for alternating weeks based on your feedback.
>>> >
>>> > As a starting point, I'd like to suggest the times alternate between
>>> UTC
>>> >  and 1600 (the current time).
>>>
>>> Sadly, both times don't work for me. I would propose something like 8
>>> UTC, which should work for most folks located in EU and east, or 18 UTC.
>>>
>>> Matthias
>>>
>>>
>>> ___
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On Wed Nov 26 2014 at 3:36:27 AM Thomas Goirand  wrote:

> On 11/21/2014 08:31 PM, Donald Stufft wrote:
> >
> >> On Nov 21, 2014, at 3:59 AM, Thomas Goirand  wrote:
> >>
> >>>
> >>> I'm not sure I understand the meaning behind this question. "bower
> >>> install angular" downloads a bower package called "angular".
> >>
> >> Isn't there is a simple URL that I may use with wget? I don't really
> >> want to use bower directly, I just would like to have a look to the
> >> content of the bower package.
> >
> > You can’t. Bower doesn’t have “traditional” packages where you take a
> > directory and archive it using tar/zip/whatever and then upload it to
> > some repo. Bower has a registry which maps names to git URLs and then
> > the bower CLI looks up that mapping, fetches the git repository and then
> > uses that as the input to the “look at metadata and do stuff with files”
> > part of the package manager instead of the output of an un-unarchival
> > command.
>
> Then this makes Bower a very bad candidate to "debianize" stuff. We'll
> have a moving target with a constantly changing git from upstream,
> meaning that we'll have all sorts of surprise in the gate.
>

It's no more constantly moving than any other project. Bower versions are
tied to git tags. In fact, since debian etc. usually go to the repository
rather than use release tarballs, bower *improves* things by requiring the
tags, making it easier for you to isolate the version in the repository
that you need, whereas otherwise people just have to remember to tag and
often don't :)



> Frankly, this Bower thing scares me, and I don't really understand why
> we're not continuing to use XStatic stuff, which was really convenient
> and has been proven to work during the Juno cycle.
>

We're doing this so we don't have the additional burden of creating and
maintaining the xstatic packages.


  Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] proposal: alternating weekly meeting time [doodle poll created]

[Richard Jones]

Thanks!

On Tue Nov 25 2014 at 3:15:49 AM Yves-Gwenaël Bourhis <
yves-gwenael.bour...@cloudwatt.com> wrote:

>
>
> Le 24/11/2014 04:20, Richard Jones a écrit :
> > Thanks everyone, I've closed the poll. I'm sorry to say that there's no
> > combination of two timeslots which allows everyone to attend a meeting.
> > Of the 25 respondents, the best we can do is cater for 24 of you.
> >
> > Optimising for the maximum number of attendees, the potential meeting
> > times are 2000 UTC Tuesday and 1000 UTC on one of Monday, Wednesday or
> > Friday. In all three cases the only person who has indicated they cannot
> > attend is Lifeless.
> >
> > Unfortunately, David has indicated that he can't be present at the
> > Tuesday 2000 UTC slot. Optimising for him as a required attendee for
> > both meetings means we lose an additional attendee, and gives us the
> > Wednesday 2000 UTC slot and a few options:
> >
> > - Monday, Wednesday and Thursday at 1200 UTC (Lifeless and ygbo miss)
>
> 1200 UTC is perfect for me.
> The doodle was proposing 1200 UTC to 1400 UTC and in the 2 hours
> bandwidth I can not be sure to be there. but if it's 1200 "on the spot"
> I can for sure :-)
> Since I couldn't precise this on the doodle I didn't select this slot. A
> one hour bandwidth would have allowed more precision but I understand
> you concern that the doodle would have been to long to scroll.
>
> --
> Yves-Gwenaël Bourhis
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] REST and Django

[Richard Jones]

On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S 
wrote:

>  Hi Richard,
>
>  You are right, we should put this out on the main ML, so copying thread
> out to there.  ML: FYI that this started after some impromptu IRC
> discussions about a specific patch led into an impromptu google hangout
> discussion with all the people on the thread below.
>

Thanks Travis!



> As I mentioned in the review[1], Thai and I were mainly discussing the
> possible performance implications of network hops from client to horizon
> server and whether or not any aggregation should occur server side.   In
> other words, some views  require several APIs to be queried before any data
> can displayed and it would eliminate some extra network requests from
> client to server if some of the data was first collected on the server side
> across service APIs.  For example, the launch instance wizard will need to
> collect data from quite a few APIs before even the first step is displayed
> (I’ve listed those out in the blueprint [2]).
>
>  The flip side to that (as you also pointed out) is that if we keep the
> API’s fine grained then the wizard will be able to optimize in one place
> the calls for data as it is needed. For example, the first step may only
> need half of the API calls. It also could lead to perceived performance
> increases just due to the wizard making a call for different data
> independently and displaying it as soon as it can.
>

Indeed, looking at the current launch wizard code it seems like you
wouldn't need to load all that data for the wizard to be displayed, since
only some subset of it would be necessary to display any given panel of the
wizard.



> I tend to lean towards your POV and starting with discrete API calls and
> letting the client optimize calls.  If there are performance problems or
> other reasons then doing data aggregation on the server side could be
> considered at that point.
>

I'm glad to hear it. I'm a fan of optimising when necessary, and not
beforehand :)



> Of course if anybody is able to do some performance testing between the
> two approaches then that could affect the direction taken.
>

I would certainly like to see us take some measurements when performance
issues pop up. Optimising without solid metrics is bad idea :)


Richard


>
>  [1]
> https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py
> [2]
> https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign
>
>  -Travis
>
>   From: Richard Jones 
> Date: Wednesday, November 26, 2014 at 11:55 PM
> To: Travis Tripp , Thai Q Tran/Silicon Valley/IBM <
> tqt...@us.ibm.com>, David Lyle , Maxime Vidori <
> maxime.vid...@enovance.com>, "Wroblewski, Szymon" <
> szymon.wroblew...@intel.com>, "Wood, Matthew David (HP Cloud - Horizon)" <
> matt.w...@hp.com>, "Chen, Shaoquan" , "Farina, Matt
> (HP Cloud)" , Cindy Lu/Silicon Valley/IBM <
> c...@us.ibm.com>, Justin Pomeroy/Rochester/IBM , Neill
> Cox 
> Subject: Re: REST and Django
>
>  I'm not sure whether this is the appropriate place to discuss this, or
> whether I should be posting to the list under [Horizon] but I think we need
> to have a clear idea of what goes in the REST API and what goes in the
> client (angular) code.
>
>  In my mind, the thinner the REST API the better. Indeed if we can get
> away with proxying requests through without touching any *client code, that
> would be great.
>
>  Coding additional logic into the REST API means that a developer would
> need to look in two places, instead of one, to determine what was happening
> for a particular call. If we keep it thin then the API presented to the
> client developer is very, very similar to the API presented by the
> services. Minimum surprise.
>
>  Your thoughts?
>
>
>   Richard
>
>
> On Wed Nov 26 2014 at 2:40:52 PM Richard Jones 
> wrote:
>
>> Thanks for the great summary, Travis.
>>
>>  I've completed the work I pledged this morning, so now the REST API
>> change set has:
>>
>>  - no rest framework dependency
>> - AJAX scaffolding in openstack_dashboard.api.rest.utils
>> - code in openstack_dashboard/api/rest/
>> - renamed the API from "identity" to "keystone" to be consistent
>> - added a sample of testing, mostly for my own sanity to check things
>> were working
>>
>>  https://review.openstack.org/#/c/136676
>>
>>
>>Richard
>>
>> On Wed Nov 26 2014 at 12:18:25 PM Tripp, Travis S 
>> wrote:
>>
>>>  Hello all,
>>>
>>>  Great discussion on the REST urls today! I think that we

Re: [openstack-dev] [horizon] REST and Django

[Richard Jones]

On Mon Dec 01 2014 at 4:18:42 PM Thai Q Tran  wrote:

> I agree that keeping the API layer thin would be ideal. I should add that
> having discrete API calls would allow dynamic population of table. However,
> I will make a case where it *might* be necessary to add additional APIs.
> Consider that you want to delete 3 items in a given table.
>
> If you do this on the client side, you would need to perform: n * (1 API
> request + 1 AJAX request)
> If you have some logic on the server side that batch delete actions: n *
> (1 API request) + 1 AJAX request
>
> Consider the following:
> n = 1, client = 2 trips, server = 2 trips
> n = 3, client = 6 trips, server = 4 trips
> n = 10, client = 20 trips, server = 11 trips
> n = 100, client = 200 trips, server 101 trips
>
> As you can see, this does not scale very well something to consider...
>
Yep, though in the above cases the client is still going to be hanging,
waiting for those server-backend calls, with no feedback until it's all
done. I would hope that the client-server call overhead is minimal, but I
guess that's probably wishful thinking when in the land of random Internet
users hitting some provider's Horizon :)

So yeah, having mulled it over myself I agree that it's useful to have
batch operations implemented in the POST handler, the most common operation
being DELETE.

Maybe one day we could transition to a batch call with user feedback using
a websocket connection.


 Richard

> [image: Inactive hide details for Richard Jones ---11/27/2014 05:38:53
> PM---On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S  Jones ---11/27/2014 05:38:53 PM---On Fri Nov 28 2014 at 5:58:00 AM Tripp,
> Travis S  wrote:
>
> From: Richard Jones 
> To: "Tripp, Travis S" , OpenStack List <
> openstack-dev@lists.openstack.org>
> Date: 11/27/2014 05:38 PM
> Subject: Re: [openstack-dev] [horizon] REST and Django
> --
>
>
>
>
> On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <*travis.tr...@hp.com*
> > wrote:
>
>Hi Richard,
>
>You are right, we should put this out on the main ML, so copying
>thread out to there.  ML: FYI that this started after some impromptu IRC
>discussions about a specific patch led into an impromptu google hangout
>discussion with all the people on the thread below.
>
>
> Thanks Travis!
>
>
>
>As I mentioned in the review[1], Thai and I were mainly discussing the
>possible performance implications of network hops from client to horizon
>server and whether or not any aggregation should occur server side.   In
>other words, some views  require several APIs to be queried before any data
>can displayed and it would eliminate some extra network requests from
>client to server if some of the data was first collected on the server side
>across service APIs.  For example, the launch instance wizard will need to
>collect data from quite a few APIs before even the first step is displayed
>(I’ve listed those out in the blueprint [2]).
>
>The flip side to that (as you also pointed out) is that if we keep the
>API’s fine grained then the wizard will be able to optimize in one place
>the calls for data as it is needed. For example, the first step may only
>need half of the API calls. It also could lead to perceived performance
>increases just due to the wizard making a call for different data
>independently and displaying it as soon as it can.
>
>
> Indeed, looking at the current launch wizard code it seems like you
> wouldn't need to load all that data for the wizard to be displayed, since
> only some subset of it would be necessary to display any given panel of the
> wizard.
>
>
>
>I tend to lean towards your POV and starting with discrete API calls
>and letting the client optimize calls.  If there are performance problems
>or other reasons then doing data aggregation on the server side could be
>considered at that point.
>
>
> I'm glad to hear it. I'm a fan of optimising when necessary, and not
> beforehand :)
>
>
>
>Of course if anybody is able to do some performance testing between
>the two approaches then that could affect the direction taken.
>
>
> I would certainly like to see us take some measurements when performance
> issues pop up. Optimising without solid metrics is bad idea :)
>
>
> Richard
>
>
>
>
>[1]
>
> *https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py*
>
> <https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py>
>[2]
>*https://blueprints.launchpad.net/horizon/+spec/launc

[openstack-dev] [Horizon] REST API split and new requirements.txt

[Richard Jones]

Hi all, just to let you know that on request from Thai I split the REST API
change into two:

https://review.openstack.org/#/c/136676
  - original change which just has the base code

https://review.openstack.org/#/c/139532
  - keystone specifics

The identity WIP should now base itself off the second change.

I've also submitted a change (https://review.openstack.org/#/c/139284/)
with the new angular dependencies to go in requirements.txt, based on a
couple of xstatic packages I created today. Also created a change for
global-requirements as well. Eventually those will be superseded by bower,
but for now it's xstatic. If anyone would like to be added to PyPI to be
able to update those xstatic packages, please ask.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] REST and Django

[Richard Jones]

Sorry I didn't respond to this earlier today, I had intended to.

What you're describing isn't REST, and the principles of REST are what have
been guiding the design of the new API so far. I see a lot of value in
using REST approaches, mostly around clarity of the interface.

While the idea of a very thin proxy seemed like a great idea at one point,
my conversations at the summit convinced me that there was value in both
using the client interfaces present in the openstack_dashboard/api code
base (since they abstract away many issues in the apis including across
versions) and also value in us being able to clean up (for example, using
"project_id" rather than "project" in the user API we've already
implemented) and extend those interfaces (to allow batched operations).

We want to be careful about what we expose in Horizon to the JS clients
through this API. That necessitates some amount of code in Horizon. About
half of the current API for keysone represents that control (the other half
is docstrings :)


 Richard


On Tue Dec 09 2014 at 9:37:47 PM Tihomir Trifonov 
wrote:

> Sorry for the late reply, just few thoughts on the matter.
>
> IMO the REST middleware should be as thin as possible. And I mean thin in
> terms of processing - it should not do pre/post processing of the requests,
> but just unpack/pack. So here is an example:
>
> instead of making AJAX calls that contain instructions:
>
> ​​
>> POST --json --data {"action": "delete", "data": [ {"name":
>> "item1"}, {"name": "item2"}, {"name": "item3" ]}
>
>
> I think a better approach is just to pack/unpack batch commands, and leave
> execution to the frontend/backend and not middleware:
>
>
> ​​
>> POST --json --data {"
>> ​batch
>> ":
>> ​[
>> {​
>> "
>> ​
>> action"
>> ​ : "delete"​
>> ,
>> ​"payload": ​
>> {"name": "item1"}
>> ​,
>> {​
>> "
>> ​
>> action"
>> ​ : "delete"​
>> ,
>> ​
>> "payload":
>> ​
>> {"name": "item
>> ​2
>> "}
>> ​,
>> {​
>> "
>> ​
>> action"
>> ​ : "delete"​
>> ,
>> ​
>> "payload":
>> ​
>> {"name": "item
>> ​3
>> "}
>> ​ ] ​
>> ​
>> ​
>> }
>
>
> ​The idea is that the middleware should not know the actual data. It
> should ideally just unpack the data:
>
> ​​responses = []
>>
>
> for cmd in
>> ​ ​
>> ​
>> ​
>> request.POST['batch']:​
>
>
>> ​
>> ​​responses
>> ​.append(​
>> ​
>> getattr(controller, cmd['action']
>> ​)(**
>> cmd['​payload']
>> ​)​)
>>
>
>> ​return responses​
>>
>
>
> ​and the frontend(JS) will just send batches of simple commands, and will
> receive a list of responses for each command in the batch. The error
> handling will be done in the frontend​(JS) as well.
>
> ​
>
> For the more complex example of 'put()' where we have dependent objects:
>
> project = api.keystone.tenant_get(request, id)
>> kwargs = self._tenant_kwargs_from_DATA(request.DATA, enabled=None)
>> api.keystone.tenant_update(request, project, **kwargs)
>
>
>
> In practice the project data should be already present in the
> frontend(assuming that we already loaded it to render the project
> form/view), so
>
> ​
> ​
> POST --json --data {"
> ​batch
> ":
> ​[
> {​
> "
> ​
> action"
> ​ : "tenant_update"​
> ,
> ​"payload": ​
> {"project": js_project_object.id, "name": "some name", "prop1": "some
> prop", "prop2": "other prop, etc."}
> ​
> ​ ] ​
> ​
> ​
> }​
>
> So in general we don't need to recreate the full state on each REST call,
> if we make the Frontent full-featured application. This way - the frontend
> will construct the object, will hold the cached value, and will just send
> the needed requests as single ones or in batches, will receive the response
> from the API backend, and will render the results. The whole processing
> logic will be held in the Frontend(JS), while the middleware will just act
> as proxy(un/packer). This way we will maintain just the logic in the
> frontend, and will not need to duplicate some logic in the middleware.
>
>
>
>
> On Tue, Dec 2, 2014 at 4:45 PM, Adam Young

Re: [openstack-dev] Moving _conf and _scripts to dashboard

[Richard Jones]

+1 to moving application configuration to the application, out of the
library.


 Richard

On Thu Dec 11 2014 at 10:38:20 AM Thai Q Tran  wrote:

> The way we are structuring our javascripts today is complicated. All of
> our static javascripts reside in /horizon/static and are imported through
> _conf.html and _scripts.html. Notice that there are already some panel
> specific javascripts like: horizon.images.js, horizon.instances.js,
> horizon.users.js. They do not belong in horizon. They belong in
> openstack_dashboard because they are specific to a panel.
>
> Why am I raising this issue now? In Angular, we need controllers written
> in javascript for each panel. As we angularize more and more panels, we
> need to store them in a way that make sense. To me, it make sense for us to
> move _conf and _scripts to openstack_dashboard. Or if this is not possible,
> then provide a mechanism to override them in openstack_dashboard.
>
> Thoughts?
> Thai
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] REST and Django

[Richard Jones]

On Fri Dec 12 2014 at 1:06:08 PM Tripp, Travis S 
wrote:

> ​>>Do we really need the lines:​
>
> >> project = api.keystone.tenant_get(request, id)
> >> kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
> ​
> I agree that if you already have all the data it is really bad to have to
> do another call. I do think there is room for discussing the reasoning,
> though.
> As far as I can tell, they do this so that if you are updating an entity,
> you have to be very specific about the fields you are changing. I actually
> see this as potentially a protectionary measure against data loss and a
> sometimes very nice to have feature. It perhaps was intended to *help*
> guard against race conditions *sometimes*.
>

Yep, it looks like I broke this API by implementing it the way I did, and
I'll alter the API so that you pass both the "current" object (according to
the client) and the parameters to alter.

Thanks everyone for the great reviewing!


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Richard Jones]

This is a good proposal, though I'm unclear on how the static_settings.py
file is populated by a developer (as opposed to a packager, which you
described).


 Richard

On Fri Dec 19 2014 at 12:59:37 AM Radomir Dopieralski <
openst...@sheep.art.pl> wrote:

> Hello,
>
> revisiting the package management for the Horizon's static files again,
> I would like to propose a particular solution. Hopefully it will allow
> us to both simplify the whole setup, and use the popular tools for the
> job, without losing too much of benefits of our current process.
>
> The changes we would need to make are as follows:
>
> * get rid of XStatic entirely;
> * add to the repository a configuration file for Bower, with all the
> required bower packages listed and their versions specified;
> * add to the repository a static_settings.py file, with a single
> variable defined, STATICFILES_DIRS. That variable would be initialized
> to a list of pairs mapping filesystem directories to URLs within the
> /static tree. By default it would only have a single mapping, pointing
> to where Bower installs all the stuff by default;
> * add a line "from static_settings import STATICFILES_DIRS" to the
> settings.py file;
> * add jobs both to run_tests.sh and any gate scripts, that would run Bower;
> * add a check on the gate that makes sure that all direct and indirect
> dependencies of all required Bower packages are listed in its
> configuration files (pretty much what we have for requirements.txt now);
>
> That's all. Now, how that would be used.
>
> 1. The developers will just use Bower the way they would normally use
> it, being able to install and test any of the libraries in any versions
> they like. The only additional thing is that they would need to add any
> additional libraries or changed versions to the Bower configuration file
> before they push their patch for review and merge.
>
> 2. The packagers can read the list of all required packages from the
> Bower configuration file, and make sure they have all the required
> libraries packages in the required versions.
>
> Next, they replace the static_settings.py file with one they have
> prepared manually or automatically. The file lists the locations of all
> the library directories, and, in the case when the directory structure
> differs from what Bower provides, even mappings between subdirectories
> and individual files.
>
> 3. Security patches need to go into the Bower packages directly, which
> is good for the whole community.
>
> 4. If we aver need a library that is not packaged for Bower, we will
> package it just as we had with the XStatic packages, only for Bower,
> which has much larger user base and more chance of other projects also
> using that package and helping with its testing.
>
> What do you think? Do you see any disastrous problems with this system?
> --
> Radomir Dopieralski
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Richard Jones]

So just to be clear, as developers we:

1. have a bower.json listing the bower component to use,
2. use bower to fetch and install those to the bower_components directory
at the top level of the Horizon repos checkout, and
3. manually edit static_settings.py when we add a new bower component to
bower.json so it knows the appropriate static files to load from that
component.

Is that correct?

The above will increase the burden on those adding or upgrading bower
components (they'll need to check the bower.json in the component for the
appropriate static files to link in) but will make life easier for the
re-packagers since they'll know which files they need to cater for in
static_settings.py


  Richard


On Mon Dec 22 2014 at 8:24:03 PM Radomir Dopieralski 
wrote:

> On 20/12/14 21:25, Richard Jones wrote:
> > This is a good proposal, though I'm unclear on how the
> > static_settings.py file is populated by a developer (as opposed to a
> > packager, which you described).
>
> It's not, the developer version is included in the repository, and
> simply points to where Bower is configured to put the files.
>
> --
> Radomir Dopieralski
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Richard Jones]

On Mon Jan 05 2015 at 7:59:14 PM Radomir Dopieralski 
wrote:

> On 05/01/15 00:35, Richard Jones wrote:
> > On Mon Dec 22 2014 at 8:24:03 PM Radomir Dopieralski
> > mailto:openst...@sheep.art.pl>> wrote:
> >
> > On 20/12/14 21:25, Richard Jones wrote:
> > > This is a good proposal, though I'm unclear on how the
> > > static_settings.py file is populated by a developer (as opposed to
> a
> > > packager, which you described).
> >
> > It's not, the developer version is included in the repository, and
> > simply points to where Bower is configured to put the files.
> > So just to be clear, as developers we:
> >
> > 1. have a bower.json listing the bower component to use,
> > 2. use bower to fetch and install those to the bower_components
> > directory at the top level of the Horizon repos checkout, and
> > 3. manually edit static_settings.py when we add a new bower component to
> > bower.json so it knows the appropriate static files to load from that
> > component.
> >
> > Is that correct?
> >
> > The above will increase the burden on those adding or upgrading bower
> > components (they'll need to check the bower.json in the component for
> > the appropriate static files to link in) but will make life easier for
> > the re-packagers since they'll know which files they need to cater for
> > in static_settings.py
>
> Well, I expect you can tell Bower to put the files somewhere else than
> in the root directory of the project -- a directory like ``bower_files``
> or something (that directory is also added to ``.gitignore`` so that you
> don't commit it by mistake). Then only that directory needs to be added
> to the ``static_settings.py``. Of course, you still need to make all the
> ``

Re: [openstack-dev] [horizon] static files handling, bower/

[Richard Jones]

I think the only outstanding question is how developers and non-packagers
populate the bower_components directory - that is, how is bower expected to
be available for them?

I think following the Storyboard approach is a good idea: isolate a
known-working node/bower environment local to horizon which is managed by
tox - so to invoke bower you run "tox -e bower ". No worries about
system installation or compatibility, and works in the gate.

Horizon installation (whenever a pip install would be invoked) would then
also have a "tox -e bower install" invocation.

Storyboard[1] uses a thing called nodeenv[2] which is installed through pip
/ requirements.txt to control the node environment. It then has bower
commands in tox.ini[3] (though I'd just have a single "bower" environment
to implement the tox command I suggest above.


 Richard

[1] https://wiki.openstack.org/wiki/StoryBoard
[2] https://pypi.python.org/pypi/nodeenv
[3]
https://git.openstack.org/cgit/openstack-infra/storyboard-webclient/tree/tox.ini


On Tue Jan 06 2015 at 11:42:17 AM Tripp, Travis S 
wrote:

> What Radomir proposes looks like it would greatly ease the process I am
> still going through to get the latest angular available to Horizon for
> current development.  At the time of writing this, I’m still trying to get
> the updated library through.  I hit a rather difficult workflow:
>
>
>   1.  Packaged the latest into Xstatic-Angular-1.3.7
>   2.  Submitted patch which deprecated the separate older
> xstatic-angular-cookies and xstatic-angular-mock packages
>   3.  Reviewed and approved (after correcting an initial mis-repackaging)
>   4.  Radomir released to Pypi
>
> This was pretty easy and not too hard. Not too much to complain about.
>
> However, now, to get Horizon to use it, I have to get that into global
> requirements.  Since I’m deprecating old packages I got stuck in a sort of
> ugly dependency path.  I couldn’t remove the cookies and mock libraries
> from the global requirements patch that added the new 1.3.7 package because
> of horizon still referencing the deprecated packages.  And, when I did it
> anyway, the integration tests failed due to horizon being dependent on
> something not in global requirements.  So, now, as far as I can tell we
> have to jump through the following hoops:
>
>
>   1.  Global requirements patch to add angular 1.3.7
>  *   Verify check / recheck fun
>  *   Reviewed and approved
>  *   Gate check / recheck fun
>   2.  Horizon patch to update to angular 1.3.7 and remove deprecated mock
> and cookies packages
>  *   Verify check / recheck fun
>  *   Reviewed and approved
>  *   Gate check / recheck fun
>   3.  Global requirements patch to remove deprecated mock and cookies
>  *   Verify check / recheck fun
>  *   Reviewed and approved
>  *   Gate check / recheck fun
>
> Don’t get me wrong, I really do think the gate is brilliant and am all for
> a review / approval process, but this does seem excessive for a UI library
> that should only be used by Horizon. Is there some other reason that this
> should have to go through global requirements?
>
> Thanks,
> Travis
>
> From: Richard Jones mailto:r1chardj0...@gmail.com>
> >
> Reply-To: OpenStack List  openstack-dev@lists.openstack.org>>
> Date: Monday, January 5, 2015 at 2:08 AM
> To: OpenStack List mailto:openstack
> -d...@lists.openstack.org>>
> Subject: Re: [openstack-dev] [horizon] static files handling, bower/
>
>
>
> On Mon Jan 05 2015 at 7:59:14 PM Radomir Dopieralski <
> openst...@sheep.art.pl<mailto:openst...@sheep.art.pl>> wrote:
> On 05/01/15 00:35, Richard Jones wrote:
> > On Mon Dec 22 2014 at 8:24:03 PM Radomir Dopieralski
> > mailto:openst...@sheep.art.pl>  openst...@sheep.art.pl<mailto:openst...@sheep.art.pl>>> wrote:
> >
> > On 20/12/14 21:25, Richard Jones wrote:
> > > This is a good proposal, though I'm unclear on how the
> > > static_settings.py file is populated by a developer (as opposed to
> a
> > > packager, which you described).
> >
> > It's not, the developer version is included in the repository, and
> > simply points to where Bower is configured to put the files.
> > So just to be clear, as developers we:
> >
> > 1. have a bower.json listing the bower component to use,
> > 2. use bower to fetch and install those to the bower_components
> > directory at the top level of the Horizon repos checkout, and
> > 3. manually edit static_settings.py when we add a new bower component to
> > bower.json so it knows the appropriate static files to load from that
> > component.
> >
> > 

Re: [openstack-dev] [horizon] static files handling, bower/

[Richard Jones]

Thanks, Radomir. How much detail from this discussion should be captured in
the blueprint? I'm afraid I'm more familiar with the Python PEP process.

On Thu Jan 08 2015 at 11:38:57 PM Radomir Dopieralski <
openst...@sheep.art.pl> wrote:

> On 06/01/15 01:53, Richard Jones wrote:
> > I think the only outstanding question is how developers and
> > non-packagers populate the bower_components directory - that is, how is
> > bower expected to be available for them?
> >
> > I think following the Storyboard approach is a good idea: isolate a
> > known-working node/bower environment local to horizon which is managed
> > by tox - so to invoke bower you run "tox -e bower ". No worries
> > about system installation or compatibility, and works in the gate.
> >
> > Horizon installation (whenever a pip install would be invoked) would
> > then also have a "tox -e bower install" invocation.
> >
> > Storyboard[1] uses a thing called nodeenv[2] which is installed through
> > pip / requirements.txt to control the node environment. It then has
> > bower commands in tox.ini[3] (though I'd just have a single "bower"
> > environment to implement the tox command I suggest above.
> >
> >
> >  Richard
> >
> > [1] https://wiki.openstack.org/wiki/StoryBoard
> > [2] https://pypi.python.org/pypi/nodeenv
> > [3] https://git.openstack.org/cgit/openstack-infra/
> storyboard-webclient/tree/tox.ini
> >
>
> I created a blueprint for this.
> https://blueprints.launchpad.net/horizon/+spec/static-file-bower
> --
> Radomir Dopieralski
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Richard Jones]

On Wed Jan 21 2015 at 7:00:12 AM Matthew Farina  wrote:

> Radomir, maybe you can help me better understand where this would go. I
> have a few questions.
>
> First, can you point me to a time when horizon used system packages
> successfully for JavaScript libraries? When I looked through the Debian and
> Ubuntu packages I couldn't find the libraries horizon is using. I'm curious
> to see this in action.
>

I'm not a user of these packaging systems, but I found this, which I
believe shows the javascript system packages used:

https://packages.debian.org/sid/openstack-dashboard



> Front-end systems almost never use system packagers like this. Can you
> point me to applications like horizon that use system packages this way? If
> Horizon is going to go it virtually alone in this space, what will that
> mean for our level of work and ability to have updates?
>

Horizon is not a typical web front end system, it's part of OpenStack, and
thus has to abide by OpenStack packaging conventions. Those dictate that we
must have Horizon packaged by system packagers for deployment.


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Richard Jones]

On Fri Jan 23 2015 at 4:28:59 AM Matthew Farina  wrote:

> I would like to add one more nuance to this discussion that I don't
> remember seeing.
>
> JavaScript libraries run in web browser in their JavaScript engines (like
> v8) rather than on the server. A version of a JS library may be fine on a
> system, without any security issues, but contain browser issues. The
> version used matters more to the application and the web browsers consuming
> the application than to the system it's on.
>

I think you're confusing Javascript in the browser vs. Javascript for
node.js

Those are two separate things. Javascript used in the browser is rarely
(never in our case) used also for node.js development. There are even two
separate packaging systems: bower is all about components for browsers,
whereas npm is all about packages for node.js. Our discussion is about
bower, and definitely not about npm or the node.js programming language.



> Some of the libraries exist as packages. For example, there are some
> debian packages. These have older versions of libraries than those that
> will work in Horizon.
>

Thanks to packagers working with OpenStack we are able to get newer
versions of the components we need once we have pinned them in our code.



> The libraries need to integrate for horizon and the browsers. So,
> supporting varying versions of a js library, their interactions together,
> and creating a usable interface will be a real problem. For example the
> debian packages of old or varying versions will a problem for those of us
> attempting to craft a UI. What's there isn't practically usable today. Some
> things are missing.
>

It's entirely up to us to specify a pinned set of component versions that
we need, which we then need the system packagers to ensure are available as
deb or rpm. Gaps will be filled as part of the usual packaging efforts that
go on during the OpenStack release cycle.


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] concrete proposal on changing the library testing model with devstack-gate

[Richard Jones]

On 30 September 2014 08:14, Doug Hellmann  wrote:

>
> On Sep 29, 2014, at 4:22 PM, Robert Collins 
> wrote:
>
> > As far as pip goes, you may not know, but tox defaults to pip --pre,
> > which means anyone using tox, like us all here, will be pulling the
> > alphas down by default: so impacting folk doing e.g. devtest on
> > icehouse. So I don't think the hack is working as intended.
>
> I dislike tox more every day. Is this why we have the installation command
> override set in tox.ini?
>

Yes. This issue in the tox tracker has some background - feel free to
upvote it!

https://bitbucket.org/hpk42/tox/issue/193/remove-the-pre-pip-option-by-default


Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

Hi all,

At the summit last week, we developed a plan for moving forward with
modernising Horizon's UI using AngularJS. If you weren't at that meeting
and are interested in helping out with this effort please let me know!

The relevant etherpad from the meeting:
https://etherpad.openstack.org/p/kilo-horizon-contributors-meetup

TL;DR: piece by piece we will replace Django views in Horizon with angular
views, and we're going to start with Identity

First up, I'd like to ask the UX folk who raised their hands in that
meeting to indicate which of the Identity panes we should start with. I
believe a wizard was mentioned, as a way to exercise the new wizard code
from Maxime.

At the same time, I'm looking at updating the AngularJS recommendations in
the wiki. I believe other aspects of the current approach to angular code
should also be revisited, if we're to scale up to the full angular
front-end envisaged. I'd appreciate if those interested in this aspect in
particular could contact me so we can sort this out as a team!

I'd like to start the design work for the new REST API layer we'll be
exposing to the angular application code, but that is also part of the
broader discussion about the structure of the angular code in the Horizon
application as mentioned above. Should it be a new blueprint/spec?

There were some discussions around tooling. We're using xstatic to manage
3rd party components, but there's a lot missing from that environment. I
hesitate to add supporting xstatic components on to the already large pile
of work we have to do, so would recommend we switch to managing those
components with bower instead. For reference the list of 3rd party
components I used in angboard* (which is really only a teensy fraction of
the total application we'd end up with, so this components list is probably
reduced):

json3
es5-shim
angular
angular-route
angular-cookies
angular-animate
angular-sanitize
angular-smart-table
angular-local-storage
angular-bootstrap
angular-translate
font-awesome
boot
underscore
ng-websocket

Just looking at PyPI, it looks like only a few of those are in xstatic, and
those are out of date.

grunt provides a lot of features for developing an angular interface. In
particular LiveReload accelerates development significantly. There's a
django-livereload but it uses tiny-lr under the hood, so we're still using
a node application for LiveReload support... so it might make sense to just
use grunt. grunt provides many other features as well (wiredep integration
with bower, build facilities with ngMin, test monitoring and reload etc).

There seemed to be agreement to move to jasmine (from qunit) for writing
the tests. It's not noted in the etherpad, but I recall karma was accepted
as a given for the test runner. For those not in the meeting, angboard uses
mocha+chai for test writing, but I agreed that jasmine is acceptable, and
is already used by Storyboard (see below).

Also, phantomjs so we don't have to fire up a browser for exercising (what
should hopefully be an extensive) unit test suite.

The Storyboard project has successfully integrated these tools into the
OpenStack CI environment.


 Richard

* https://github.com/r1chardj0n3s/angboard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] proposal: alternating weekly meeting time

[Richard Jones]

Hi all,

At the summit meetup last week I proposed that the Horizon weekly meeting
time alternate between the current time and something more suitable for
those of us closer to UTC+10. I'd like to get an indication of the interest
in this, and I'll look into getting a second meeting time booked for
alternating weeks based on your feedback.

As a starting point, I'd like to suggest the times alternate between UTC
 and 1600 (the current time).


Thanks,

Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 11 November 2014 20:10, Anton Zemlyanov  wrote:

> Modernizing Horizon using Angular is really great. I would suggest a
> couple of minor things to consider.
>
> First, font-awesome is excellent and free font, but there are two
> problems: it has lots of icons that are not used, it also miss some other
> icons worth to have. I would suggest Fontello, it is tool for picking
> glyphs from all the free fonts round there and building custom fonts
>

Thanks for bringing this up. We discussed the need for additional symbols
at the meetup, and it may very well be that we do come up with a custom
font with a selection from font-awesome plus additional symbols designed as
we go along. Very good idea.



> Grunt is being forced out by Gulp that is times faster while having the
> same level of plugins and tools and is easier to use
>

Thanks for the pointer. I mentioned grunt in particular because it's quite
mature at this point, was the default tool for the yoeman project builder I
used and is also the tool of choice for Storyboard. If there is a
compelling reason to switch though, I imagine the Storyboard folk may also
be interested. From a brief look around there does seem to be a lot of
enthusiasm for gulp. I will look at making a branch of angboard that uses
gulp to get a good feeling for it (unless someone produces a PR that does
it for me ;)


 Richard


> On Tue, Nov 11, 2014 at 11:02 AM, Richard Jones 
> wrote:
>
>> Hi all,
>>
>> At the summit last week, we developed a plan for moving forward with
>> modernising Horizon's UI using AngularJS. If you weren't at that meeting
>> and are interested in helping out with this effort please let me know!
>>
>> The relevant etherpad from the meeting:
>> https://etherpad.openstack.org/p/kilo-horizon-contributors-meetup
>>
>> TL;DR: piece by piece we will replace Django views in Horizon with
>> angular views, and we're going to start with Identity
>>
>> First up, I'd like to ask the UX folk who raised their hands in that
>> meeting to indicate which of the Identity panes we should start with. I
>> believe a wizard was mentioned, as a way to exercise the new wizard code
>> from Maxime.
>>
>> At the same time, I'm looking at updating the AngularJS recommendations
>> in the wiki. I believe other aspects of the current approach to angular
>> code should also be revisited, if we're to scale up to the full angular
>> front-end envisaged. I'd appreciate if those interested in this aspect in
>> particular could contact me so we can sort this out as a team!
>>
>> I'd like to start the design work for the new REST API layer we'll be
>> exposing to the angular application code, but that is also part of the
>> broader discussion about the structure of the angular code in the Horizon
>> application as mentioned above. Should it be a new blueprint/spec?
>>
>> There were some discussions around tooling. We're using xstatic to manage
>> 3rd party components, but there's a lot missing from that environment. I
>> hesitate to add supporting xstatic components on to the already large pile
>> of work we have to do, so would recommend we switch to managing those
>> components with bower instead. For reference the list of 3rd party
>> components I used in angboard* (which is really only a teensy fraction of
>> the total application we'd end up with, so this components list is probably
>> reduced):
>>
>> json3
>> es5-shim
>> angular
>> angular-route
>> angular-cookies
>> angular-animate
>> angular-sanitize
>> angular-smart-table
>> angular-local-storage
>> angular-bootstrap
>> angular-translate
>> font-awesome
>> boot
>> underscore
>> ng-websocket
>>
>> Just looking at PyPI, it looks like only a few of those are in xstatic,
>> and those are out of date.
>>
>> grunt provides a lot of features for developing an angular interface. In
>> particular LiveReload accelerates development significantly. There's a
>> django-livereload but it uses tiny-lr under the hood, so we're still using
>> a node application for LiveReload support... so it might make sense to just
>> use grunt. grunt provides many other features as well (wiredep integration
>> with bower, build facilities with ngMin, test monitoring and reload etc).
>>
>> There seemed to be agreement to move to jasmine (from qunit) for writing
>> the tests. It's not noted in the etherpad, but I recall karma was accepted
&g

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 11 November 2014 20:53, Jiri Tomasek  wrote:

>  Hey,
>
> Thanks for writing this up!
>
> I am including some notes and questions inline...
>
> On 11/11/2014 08:02 AM, Richard Jones wrote:
>
> Hi all,
>
>  At the summit last week, we developed a plan for moving forward with
> modernising Horizon's UI using AngularJS. If you weren't at that meeting
> and are interested in helping out with this effort please let me know!
>
>  The relevant etherpad from the meeting:
> https://etherpad.openstack.org/p/kilo-horizon-contributors-meetup
>
>  TL;DR: piece by piece we will replace Django views in Horizon with
> angular views, and we're going to start with Identity
>
>  First up, I'd like to ask the UX folk who raised their hands in that
> meeting to indicate which of the Identity panes we should start with. I
> believe a wizard was mentioned, as a way to exercise the new wizard code
> from Maxime.
>
>  At the same time, I'm looking at updating the AngularJS recommendations
> in the wiki. I believe other aspects of the current approach to angular
> code should also be revisited, if we're to scale up to the full angular
> front-end envisaged. I'd appreciate if those interested in this aspect in
> particular could contact me so we can sort this out as a team!
>
> I am interested.
>

Excellent!



> I'd like to start the design work for the new REST API layer we'll be
> exposing to the angular application code, but that is also part of the
> broader discussion about the structure of the angular code in the Horizon
> application as mentioned above. Should it be a new blueprint/spec?
>
>
> I think spec seems appropriate.  Do you think using django-angular would
> be convenient?
>

As I understand it, django-angular exists to make it easier to integrate
Django views and angular views, whereas what we're after is for them to be
as separate as possible, with Django providing a single templated page that
forms the base page for the angular single-page app, and the REST API is as
un-Django as possible (using something like tastypie).




> There were some discussions around tooling. We're using xstatic to manage
> 3rd party components, but there's a lot missing from that environment. I
> hesitate to add supporting xstatic components on to the already large pile
> of work we have to do, so would recommend we switch to managing those
> components with bower instead. For reference the list of 3rd party
> components I used in angboard* (which is really only a teensy fraction of
> the total application we'd end up with, so this components list is probably
> reduced):
>
>  json3
> es5-shim
> angular
> angular-route
> angular-cookies
> angular-animate
> angular-sanitize
> angular-smart-table
>  angular-local-storage
> angular-bootstrap
> angular-translate
> font-awesome
> boot
> underscore
> ng-websocket
>
>  Just looking at PyPI, it looks like only a few of those are in xstatic,
> and those are out of date.
>
>  grunt provides a lot of features for developing an angular interface. In
> particular LiveReload accelerates development significantly. There's a
> django-livereload but it uses tiny-lr under the hood, so we're still using
> a node application for LiveReload support... so it might make sense to just
> use grunt. grunt provides many other features as well (wiredep integration
> with bower, build facilities with ngMin, test monitoring and reload etc).
>
>  There seemed to be agreement to move to jasmine (from qunit) for writing
> the tests. It's not noted in the etherpad, but I recall karma was accepted
> as a given for the test runner. For those not in the meeting, angboard uses
> mocha+chai for test writing, but I agreed that jasmine is acceptable, and
> is already used by Storyboard (see below).
>
>  Also, phantomjs so we don't have to fire up a browser for exercising
> (what should hopefully be an extensive) unit test suite.
>
>  The Storyboard project has successfully integrated these tools into the
> OpenStack CI environment.
>
>
> Using javascript tooling (yeoman, grunt, bower, etc.) has this issue of
> being dependent on nodejs which if I recall correctly is causing problems
> for packagers as some versions of these tools require different nodejs
> versions - please Mathias correct me if I am wrong. I know this discussion
> has been here before, but using these tools is necessary for effective
> development. So we need to resolve the problem asap. Storyboard does not
> have this issue as it is infra thing.
>

Infra still did have to do work to make things reliably build, and pa

Re: [openstack-dev] URLs

[Richard Jones]

On 12 November 2014 09:47, Ian Cordasco  wrote:

> On 11/11/14, 16:35, "Adam Young"  wrote:
>
> >Recent recurrence of the "Why ios everything on its own port" question
> >triggered my desire to take this pattern and put it to rest.
> >
> >My suggestion, from a while ago, was to have a naming scheme that
> >deconflicts putting all of the services onto a single server, on port 443.
> >
> >I've removed a lot of the cruft, but not added in entries for all the
> >new *aaS services.
> >
> >
> >https://wiki.openstack.org/wiki/URLs
> >
> >Please add in anything that should be part of OpenStack.  Let's make
> >this a reality, and remove the  specific ports.
>

What do those URLs map to? For example, what does
https://hostname/identity/main map to, and does this scheme intend to
handle multiple endpoints per service?



> Is the reason subdomains aren’t being suggested because of the expense of
> *.hostname certificates?
>

Well, it's also much more effort even without certs :)
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] proposal: alternating weekly meeting time [doodle poll created]

[Richard Jones]

I have set up a doodle poll to let folk enter their preferred times. It's
in UTC/GMT (/London time, because doodle) so use something like
http://everytimezone.com/ to figure that out :)

https://doodle.com/47h3f35nad62ncnf


 Richard

On 11 November 2014 18:46, Matthias Runge  wrote:

> On 11/11/14 08:09, Richard Jones wrote:
> > Hi all,
> >
> > At the summit meetup last week I proposed that the Horizon weekly
> > meeting time alternate between the current time and something more
> > suitable for those of us closer to UTC+10. I'd like to get an indication
> > of the interest in this, and I'll look into getting a second meeting
> > time booked for alternating weeks based on your feedback.
> >
> > As a starting point, I'd like to suggest the times alternate between UTC
> >  and 1600 (the current time).
>
> Sadly, both times don't work for me. I would propose something like 8
> UTC, which should work for most folks located in EU and east, or 18 UTC.
>
> Matthias
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 12 November 2014 18:17, Matthias Runge  wrote:

> On 11/11/14 10:53, Jiri Tomasek wrote:
> > Hey,
> >
> > Thanks for writing this up!
>
> >> The Storyboard project has successfully integrated these tools into
> >> the OpenStack CI environment.
>
> OpenStack CI and distributors are different, because OpenStack CI does
> not distribute software.
>

Ah, I wasn't clear; my concern was whether the tools chosen would be
compatible with the CI environment. I'm hoping that distribution of the
tools isn't our concern (see below).



> > Using javascript tooling (yeoman, grunt, bower, etc.) has this issue of
> > being dependent on nodejs which if I recall correctly is causing
> > problems for packagers as some versions of these tools require different
> > nodejs versions - please Mathias correct me if I am wrong. I know this
> > discussion has been here before, but using these tools is necessary for
> > effective development. So we need to resolve the problem asap.
> > Storyboard does not have this issue as it is infra thing.
>
> As far as I know, those tools don't require different nodejs versions.
> But: we can not have different node.js versions installed at the same
> time. I assume, this is true for all distributions. Creating and
> maintaining parallel installable versions just sucks and causes many
> issues.
>

I believe the nodeenv method of installing node solves this, as it's
entirely local to the development environment.


I will have to go through all dependencies and do a review, if those are
> acceptable for inclusion e.g in Fedora. The same is true for Thomas
> Goirand for inclusion in Debian.
>
> >
> > Petr Belanyi has added optional jshint install for js linting into
> > Horizon and it installs nodejs as it depends on it. Could this approach
> > work for our need of js tooling too? [1]
>
> Sigh, this nonsense doesn't go away? This is the third time the same
> issue comes up.
>
> jshint is NOT free software.
>
> https://github.com/jshint/jshint/blob/master/src/jshint.js#L19
> 


They're trying to resolve that https://github.com/jshint/jshint/issues/1234

But regardless, jshint doesn't have to be installed from a Linux
repository; it's usually installed using npm alongside the other node tools.


Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] proposal: alternating weekly meeting time [doodle poll created]

[Richard Jones]

To clarify - I created the doodle in UTC (so the times in there are UTC
00:00 Monday to 23:59 Friday) but they will be shifted to your local
timezone - scroll all the way to the right to double-check that it's set
correctly for you.

Sorry for the confusion.


 Richard

On 12 November 2014 12:45, Richard Jones  wrote:

> I have set up a doodle poll to let folk enter their preferred times. It's
> in UTC/GMT (/London time, because doodle) so use something like
> http://everytimezone.com/ to figure that out :)
>
> https://doodle.com/47h3f35nad62ncnf
>
>
>  Richard
>
> On 11 November 2014 18:46, Matthias Runge  wrote:
>
>> On 11/11/14 08:09, Richard Jones wrote:
>> > Hi all,
>> >
>> > At the summit meetup last week I proposed that the Horizon weekly
>> > meeting time alternate between the current time and something more
>> > suitable for those of us closer to UTC+10. I'd like to get an indication
>> > of the interest in this, and I'll look into getting a second meeting
>> > time booked for alternating weeks based on your feedback.
>> >
>> > As a starting point, I'd like to suggest the times alternate between UTC
>> >  and 1600 (the current time).
>>
>> Sadly, both times don't work for me. I would propose something like 8
>> UTC, which should work for most folks located in EU and east, or 18 UTC.
>>
>> Matthias
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 13 November 2014 09:35, Adam Young  wrote:

> On 11/12/2014 03:03 AM, Matthias Runge wrote:
>
>> On 12/11/14 08:40, Richard Jones wrote:
>>
>>  I believe the nodeenv method of installing node solves this, as it's
>>> entirely local to the development environment.
>>>
>> See below, this touches package build as well.
>>
>>>
>>>  I will have to go through all dependencies and do a review, if
>>> those are
>>>  acceptable for inclusion e.g in Fedora. The same is true for Thomas
>>>  Goirand for inclusion in Debian.
>>>
>>>  >
>>>  > Petr Belanyi has added optional jshint install for js linting into
>>>  > Horizon and it installs nodejs as it depends on it. Could this
>>> approach
>>>  > work for our need of js tooling too? [1]
>>>
>>>  Sigh, this nonsense doesn't go away? This is the third time the same
>>>  issue comes up.
>>>
>>>  jshint is NOT free software.
>>>
>>>  https://github.com/jshint/jshint/blob/master/src/jshint.js#L19
>>>
>>>
>>> They're trying to resolve that https://github.com/jshint/
>>> jshint/issues/1234
>>>
>>> But regardless, jshint doesn't have to be installed from a Linux
>>> repository; it's usually installed using npm alongside the other node
>>> tools.
>>>
>>>  Thanks for the pointer, this is good news!
>>
>> Regarding package managers, my POV is completely different. From a
>> distributor perspective, where customers expect everything provided from
>> a single source, I'm not using npm, pip etc. I'm packaging that stuff
>> properly before using it.
>>
>> There are companies out there, where security policy does not allow to
>> install software from a third party repository. pypi etc. are considered
>> as a third party here in this context.
>>
>> I would prefer to have the complete tool chain available as (RPM)
>> packages. I am executing unit tests etc. during package build. Our
>> builders don't have access to the internet, downloading any other stuff
>> from the internet is no option.
>>
>
> And this is really not-negotiable, too.  Debian has the same requirements,
> it is not Red Hat/Fedora speciufic.  It is no different than the Python
> Code.  We dodn't pip install for RHOSP for the Python packages, and we
> can't use any of the language specific package managers.
>
> But, we are used to it, and dealing with package dependencies is what we
> do.  Having these in Fedora, while painful, will be ultimately very
> valuable.
>

I guess a point of difference here is that the Linux packaging systems
target global installation, whereas bower and npm (in our case) target a
local installation. It's effectively vendoring the code, but you still have
to pull it down from a repository for each installation. Don't worry, I'm
not going to suggest we actually move back to vendoring :)

We're currently using xstatic and that works with Linux packaging because
it was designed to cope with being a global installation. The current
Horizon codebase has a django-xstatic plugin which further makes dealing
with xstatic components nicer - for example it handles path management and
static file compilation (JS minification and concatenation, for example).
That's really nice, but poses some problems:

- we would need to xstatic-ify (and deb/rpm-ify) all those components
- we could run into global version conflict issues if we run more than one
service on a system - is this likely to be an issue in practise though?
- as far as I'm aware, the xstatic JS minification is not angular-aware,
and will break angular code that has not been written to be
dumb-minifier-aware (the angular minifier "ngMin" is written in node and
knows how to do things more correctly); adding dumb-minifier-awareness to
angular code makes it ugly and more error-prone :/

On the minification issue though: I talked to a few people at the summit
and we agreed that minification should just be avoided; gzip at the
transport layer buys significantly better compression, and has the added
bonus that the resultant code is readable. I note that the Debian JS
guidelines* only recommend that libraries *should* be minified (though I'm
unsure why they even recommend that).

One further note on xstatic vs. bower: during development it's very useful
to be able to install random components from bower to try them out; during
angboard development I would sometimes install and remove half a dozen
components just in one day. I would hope that whatever solution we come up
with has at least some of this flexibility for experimentation.


Richard

* https://wiki.debian.org/Javascript/Policy
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

Gabriel has responded saying very much what I would have said, so I won't
repeat that. I would like to note though that bower and npm are two
separate beasts entirely. The dependency trees in bower are very limited
indeed (only two additional components are installed beyond the list below)
which is in stark contrast to npm. The xstatic question only applies to the
bower components - the npm stuff is all tool chain; tools that I believe
should be packaged as such by packagers.

I would like to address one specific point Thomas raised though:

On 13 November 2014 12:29, Thomas Goirand  wrote:

> I just don't understand why all of this is needed. I did some JS
> programming myself back in the days (10 years ago, using PHP...). I
> could do Ajax by hand, without using a single library. What is it that
> you're doing in Horizon that needs so many libs? When I see Horizon in
> Juno, I don't even get it. Or is it because you just want to have fancy
> animation? Frankly, I don't care these. I care a lot more that we're not
> adding new potential security problems.
>

Horizon is an incredibly complex application. Just so we're all on the same
page, the components installed by bower for angboard are:

angular
  Because writing an application the size of Horizon without it would be
madness :)
angular-route
  Provides structure to the application through URL routing.
angular-cookies
  Provides management of browser cookies in a way that integrates well with
angular.
angular-sanitize
  Allows direct embedding of HTML into angular templates, with sanitization.
json3
  Compatibility for older browsers so JSON works.
es5-shim
  Compatibility for older browsers so Javascript (ECMAScript 5) works.
angular-smart-table
  Table management (population, sorting, filtering, pagination, etc)
angular-local-storage
   Browser local storage with cookie fallback, integrated with angular
mechanisms.
angular-bootstrap
   Extensions to angular that leverage bootstrap (modal popups, tabbed
displays, ...)
font-awesome
   Additional glyphs to use in the user interface (warning symbol, info
symbol, ...)
boot
   Bootstrap for CSS styling (this is the dependency that brings in jquery
and requirejs)
underscore
   Javascript utility library providing a ton of features Javascript lacks
but Python programmers expect.
ng-websocket
   Angular-friendly interface to using websockets
angular-translate
   Support for localization in angular using message catalogs generated by
gettext/transifex.
angular-mocks
   Mocking support for unit testing angular code
angular-scenario
   More support for angular unit tests

Additionally, angboard vendors term.js because it was very poorly packaged
in the bower ecosystem. +1 for xstatic there I guess :)

So those are the components we needed to create the prototype in a few
weeks. Not using them would have added months (or possibly years) to the
development time. Creating an application of the scale of Horizon without
leveraging all that existing work would be like developing OpenStack while
barring all use of Python 3rd-party packages.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 14 November 2014 02:04, Thomas Goirand  wrote:

> On 11/13/2014 12:13 PM, Richard Jones wrote:
> > the npm stuff is all tool chain; tools
> > that I believe should be packaged as such by packagers.
>
> npm is already in Debian:
> https://packages.debian.org/sid/npm
>
> However, just like we can't use CPAN, "pear install", "pip install" and
> such when building or installing package, we wont be able to use NPM.
> This means every single dependency that isn't in Debian will need to be
> packaged.
>

Just to be clearer, when I wrote "the npm stuff" I meant "npm and the tools
installed by it", so grunt, bower, karma, phantomjs, etc. Not the stuff
managed by bower, just the stuff installed by npm. Those npm-based things
should be packaged by the distros as tools, just like other programs the
distros package.


> Horizon is an incredibly complex application. Just so we're all on the
> > same page, the components installed by bower for angboard are:
> >
> > angular
> >   Because writing an application the size of Horizon without it would be
> > madness :)
> > angular-route
> >   Provides structure to the application through URL routing.
> > angular-cookies
> >   Provides management of browser cookies in a way that integrates well
> > with angular.
> > angular-sanitize
> >   Allows direct embedding of HTML into angular templates, with
> sanitization.
> > json3
> >   Compatibility for older browsers so JSON works.
> > es5-shim
> >   Compatibility for older browsers so Javascript (ECMAScript 5) works.
> > angular-smart-table
> >   Table management (population, sorting, filtering, pagination, etc)
> > angular-local-storage
> >Browser local storage with cookie fallback, integrated with angular
> > mechanisms.
> > angular-bootstrap
> >Extensions to angular that leverage bootstrap (modal popups, tabbed
> > displays, ...)
> > font-awesome
> >Additional glyphs to use in the user interface (warning symbol, info
> > symbol, ...)
> > boot
> >Bootstrap for CSS styling (this is the dependency that brings in
> > jquery and requirejs)
> > underscore
> >Javascript utility library providing a ton of features Javascript
> > lacks but Python programmers expect.
> > ng-websocket
> >Angular-friendly interface to using websockets
> > angular-translate
> >Support for localization in angular using message catalogs generated
> > by gettext/transifex.
> > angular-mocks
> >Mocking support for unit testing angular code
> > angular-scenario
> >More support for angular unit tests
> >
> > Additionally, angboard vendors term.js because it was very poorly
> > packaged in the bower ecosystem. +1 for xstatic there I guess :)
> >
> > So those are the components we needed to create the prototype in a few
> > weeks. Not using them would have added months (or possibly years) to the
> > development time. Creating an application of the scale of Horizon
> > without leveraging all that existing work would be like developing
> > OpenStack while barring all use of Python 3rd-party packages.
>
> I have no problem with adding dependencies. That's how things work, for
> sure, I just want to make sure it doesn't become hell, with so many
> components inter-depending on 100s of them, which would become not
> manageable. If we define clear boundaries, then fine! The above seems
> reasonable anyway.
>
> Though did you list the dependencies of the above?
>

Again, just so we're clear, yes, the above is *all* the components
installed by bower, including dependencies (jquery and requirejs being the
*only* dependencies not directly installed).

As I mentioned, the dependency trees in bower are significantly simpler
than npm packages tend to be (most bower packages have zero or one
dependency). The "100s" of dependencies are in npm packages - but as Martin
Gleiser has pointed out, npm solves that problem with its local install and
node_modules directory structures.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

I think that it boils down to whether it'is possible that distributions:

1. package the node-based tools (grunt, karma, protractor, ...) as
installable programs, and
2. xstatic-package the bower-based packages that we use (probably a couple
of dozen at least).

We might even be able to get away without using grunt, though an
alternative to its LiveReload facility (and one that doesn't then also
depend on another node program like django-livereload does) would be
required. I believe tox and django's runserver (and other manage commands)
could suffice to replace the other functionality typically provided by
grunt.


Richard

On 14 November 2014 18:51, Radomir Dopieralski 
wrote:

> On 13/11/14 23:30, Martin Geisler wrote:
>
> [...]
>
> > While I agree that it's chaotic, I also think you make the problem worse
> > than it really is. First, remember that the user who installs Horizon
> > won't need to use the JavaScript based *developer* tools such as npm,
> > bower, etc.
> >
> > That is, I think Horizon developers will use these tools to produce a
> > release -- a tarball -- and that tarball will be something you unpack on
> > your webserver and then you're done. I base this on what I've seen in
> > the project I've been working. The release tarball you download here
> > don't mention npm, bower, or any of the other tools:
> >
> >   https://github.com/zerovm/swift-browser/releases
> >
> > The tools were used to produce the tarball and were used to test it, but
> > they're not part of the released product. Somewhat similar to how GCC
> > isn't included in the tarball if you download a pre-compiled binary.
>
> [...]
>
> > Maybe a difference is that you don't (yet) install a web application
> > like you install a system application. Instead you *deploy* it: you
> > unpack files on a webserver, you configure permissions, you setup cache
> > rules, you configure a database, etc.
>
> [...]
>
> I think I see where the misunderstanding is in this whole discussion. It
> seems it revolves around the purpose and role of the distribution.
>
> From the naive point of view, the role of a Linux distribution is to
> just collect all the software from respective upstream developers and
> put it in a single repository, so that it can be easily installed by the
> users. That's not the case.
>
> The role of a distribution is to provide a working ecosystem of
> software, that is:
> a) installed and configured in consistent way,
> b) tested to work with the specific versions that it ships with,
> c) audited for security,
> d) maintained, including security patches,
> e) documented, including external tutorials and the like,
> f) supported, either by the community or by the companies that provide
> support,
> g) free of licensing issues and legal risks as much as possible,
> h) managed with the common system management tools.
>
> In order to do that, they can't just take a tarball and drop it in a
> directory. They always produce their own builds, to make sure it's the
> same thing that the source code specifies. They sometimes have to
> rearrange configuration files, to make them fit the standards in their
> system. They provide sane configuration defaults. They track the
> security reports about all the installed components, and apply fixes,
> often before the security issue is even announced.
>
> Basically, a distribution adds a whole bunch of additional guarantees
> for the software they ship. Those are often long-term guarantees, as
> they will be supporting our software long after we have forgotten about
> it already.
>
> You say that "web development doesn't work like that". That may be true,
> but that's mostly because if you develop a new web application in-house,
> and deploy it on your server, you don't really have such large legal
> risk, configuration complexity or support problem -- you just have to
> care about that single application, because the packagers from the
> distribution that you are using are taking care about all the rest of
> software on your server.
>
> --
> Radomir Dopieralski
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] proposal: alternating weekly meeting time [doodle poll created]

[Richard Jones]

I did it that way so that the form wouldn't blow out completely, and I
thought for our purposes of scheduling a 2 hour precision would be good
enough.

On 14 November 2014 22:32, Jaromir Coufal  wrote:

> On 12/11/14 02:45, Richard Jones wrote:
>
>> I have set up a doodle poll to let folk enter their preferred times.
>> It's in UTC/GMT (/London time, because doodle) so use something like
>> http://everytimezone.com/ to figure that out :)
>>
>> https://doodle.com/47h3f35nad62ncnf
>>
>>
>>   Richard
>>
>
> Quick Question:
>
> Why is the length of the time slot 2h? Since it should take just 1h,
> should I consider beginning or end of the meeting to be relevant? It is
> challenging to find time slot between other meetings, since I might have
> just 1h slot there.
>
> -- Jarda
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 15 November 2014 00:58, Radomir Dopieralski 
wrote:

> On 14/11/14 13:02, Richard Jones wrote:
> > I think that it boils down to whether it'is possible that
> > distributions:
> >
> > 1. package the node-based tools (grunt, karma, protractor, ...) as
> > installable programs, and
> > 2. xstatic-package the bower-based packages that we use (probably a
> > couple of dozen at least).
> >
> > We might even be able to get away without using grunt, though an
> > alternative to its LiveReload facility (and one that doesn't then also
> > depend on another node program like django-livereload does) would be
> > required. I believe tox and django's runserver (and other manage
> > commands) could suffice to replace the other functionality typically
> > provided by grunt.
>
> We don't really need Xstatic for that. The packagers can as well package
> the bower-based packages directly. We can use anything, really,
> as long as we follow a process that makes sure that Horizon can be
> packaged into the different distributions. That is, we need:
>

xstatic provides additional meta-data that makes it much easier to
integrate the static bundle into an application - specifically it
automatically provides the application with file locations and endpoints
needed to be inserted into the application HTML. That stuff would have to
be done manually without xstatic, and would be a configuration pain.



> 1. All dependencies explicit (with tests failing if a dependency is
>missing),
>

xstatic provides us with a dependency mechanism through standard Python
setuptools facilities.



> 2. explicit version ranges,
>

xstatic is done through requirements.txt, so yep :)



> 3. no multiple versions of the same library,
>

This is not a thing in bower/client-side land. It's really only an issue
for npm-based packages, and as I've mentioned, the things we're using there
should be packaged as tools by the linux vendor, rather than accessed as
packages by our system. Except on non-Linux, of course, where we'll just
use npm ;)

So I guess the big open question is about how distros are going to deal
with npm tool packaging. I can't really answer that question for them
(except to say: don't try to replace npm's dependency solution with your
own; it simply won't work
because you'll probably never find a set of versions that satisfy even just
the few tools we're looking at for this project).


4. additions and upgrades of libraries moderated by the packagers,
>

Is there already some mechanism for handling the creation and management of
xstatic packages and how they interact with linux packagers? Sorry if this
is a noob question.



> 5. ability to replace the development environment with packaged
>libraries from the system,
>

I would very much like to still use bower for rapid development and
testing, with xstatic coming in once the experimentation is over. But if
there was a tool to automatically generate an xstatic package from a bower
one, that would be eaiser...



> 6. ability to build and test our software with the tools that can be
>used by all the distributions.
>

Yep, I think that just implies that the xstatic stuff is done, and that the
distros package the few npm tools we use.


On 15 November 2014 01:05, Thomas Goirand  wrote:

> On 11/11/2014 03:02 PM, Richard Jones wrote:
> > json3
> > es5-shim
> > angular
> > angular-route
> > angular-cookies
> > angular-animate
> > angular-sanitize
> > angular-smart-table
> > angular-local-storage
> > angular-bootstrap
> > angular-translate
> > font-awesome
> > boot
> > underscore
> > ng-websocket
>
> Just FYI, in Debian, the libjs-angularjs already carries:
>
> angular-route.js
> angular-cookies.js
> angular-animate.js
> angular-sanitize.js
>
> We also already have packaged:
> font-awesome
> underscore
>
> So, basically, I'd have to package:
> json3
> es5-shim
> boot
> angular-smart-table
> angular-local-storage
> angular-translate
> ng-websocket
>
> That's a reasonable amount of work. Multiply this by 2 for the xstatic
> packages (if we keep using that), that's about 14 new packages.
>

The issue is integration with the Django application. How do we know what
the file path is to the entry point for that that code, and also how it
differs from other packagers? xstatic takes care of that for us (in much
the same way that bower does), so is valuable.



> By the way, can't we use libjs-sockjs instead of ng-websocket?
>

They offer different functionality, as far as I can tell. sockjs is a
compatibility layer thing, I thi

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 17 November 2014 06:42, Thomas Goirand  wrote:
>
> On 11/15/2014 06:27 AM, Gabriel Hurley wrote:
> > So, I propose that a group gets together and defines criteria:
> > we need to accept that the Horizon team (and those knowledgeable
> > about web-app development) know best what tools they need, and
> > they need to produce such a list as a starting point. We then
> > need packagers and maintainers to examine that list and evaluate
> > it for problems (non-free software, irresolvable dependencies,
> > etc.). They're looking strictly for things which are un-packageable,
> > not commenting on the necessity of said software. And we need people
> > (thanks, Monty) willing to build new tools to find a way to turn
> > that list into something the package maintainers can consume
> > without burdening either side.
> >
> > Make sense?
> >
> >  - Gabriel
>
> I'd be happy to be in this group.

As would I, hence I started the conversation in the first place :)


> Selenium
> 
> As I wrote previously, the biggest issue currently for me, is selenium.
> It is very frustrating that I can't run these unit tests when building
> package, and potentially loose the opportunity to discover problems. I
> really would like this to be solved. There's 2 ways of solving it:
>
> 1/ Someone works so that the .xpi can be built together with the rest of
> selenium, and therefore, selenium can leave the non-free repository of
> Debian and go in main (and also be uploaded in other distros).
>
> 2/ We move away from Selenium and decide to use something else like
> PhantomJS.

I think you're confusing a couple of things here. Selenium is a system for
Javascript running tests in a browser environment. To do that, it needs a
browser. PhantomJS provides a headless browser to do that. The tests end up
being faster, less intrusive on a desktop and much simpler to run on
servers (no virtual X11 shenanigans).

I advocate for using PhantomJS, but also for using a reduced Selenium suite
where possible - with an emphasis on unit testing of the angular code
directly. Selenium is just so flaky, especially with an interface that's
even slightly dynamic.


> What we care is to find a system that will satisfy both worlds:
> distributions & upstream fast moving development. It is looking like NPM
> has the best feature and that it would be a winner against Bower and
Grunt.

Again, just to be clear: npm and bower are *both* packaging systems and
have completely separate domains of use. npm is used to package
command-line tools and libraries written in the node.js programming
language whereas bower is used to package browser application components.
It's not npm-or-bower.

bower is most likely going to be replaced by xstatic in our environment,
assuming we have some simple mechanism for creating xstatic packages from
bower components.

The distributions are going to have to package up the npm-packaged tools
that we need, though that set of packages is looking smaller and smaller,
and will probably just include the testing tools (karma, protractor,
jasmin, phantomjs). Some of those are already packaged \o/


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 17 November 2014 11:11, Thomas Goirand  wrote:

> On 11/15/2014 05:03 AM, Matthias Runge wrote:
> > On 14/11/14 16:21, Adam Young wrote:
> >
> >> Example:  I don't need Grunt to run a web server.  I need Apache for
> >> that.  Grunt does not need to be in the distro, mod_wsgi does.
> >
> > I will need every tool required to run e.g. unit tests or selenium tests
> > to be packaged. Why? Because our builders don't have access to the
> > internet and I want to be sure, the packaged version of horizon still
> > passes tests.
> >
> > Matthias
>
> Except that selenium is non-free: it's in the non-free repository of
> Debian, because it contains a pre-built .xpi plugin for firefox, which
> itself contains pre-built .so and .dll files.
>

Hasn't this issue already been addressed? Horizon currently uses
Selenium-based integration tests.



> So, we're on the same page, but selenium is a bad tool... :)
>
> Chatting with Maxime Vidori on IRC, there's a bunch of alternative
> solutions that we could use instead of Selenium. PhantomJS for example
> is already in Debian main. I would very much welcome switching to that
> instead of Selenium.
>

PhantomJS isn't a testing framework - it's just a headless browser that's
scriptable from Javascript. It has a webdriver for Selenium, which is a
testing framework. If you ditch Selenium then you need to replace it with a
testing framework on top of PhantomJS.

There's also CasperJS which is an entirely separate testing framework built
over PhantomJS (and SlimerJS, which looks to be similar to PhantomJS except
built on Gecko - but not headless). It's very JUnit. Some people like that
;)

While Selenium has issues, I'm concerned that there's very few people out
there in the wilds doing interface testing without it. Indeed there's
people advocating for Selenium over PhantomJS directly. Using PhantomJS
directly means we won't have a suite of tests we can then target at IE,
Safari, etc to ensure that the interface works there in an automated
fashion. PhantomJS might be built on WebKit, but it's different enough from
Chrome that you still want to run your tests on Chrome to ensure the
interface actually works. SlimerJS does allow the testing to be slightly
broader, but we'd still miss automated IE and Safari tests.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 17 November 2014 21:54, Radomir Dopieralski 
wrote:
>
> On 17/11/14 09:53, Martin Geisler wrote:
>
> [...]
>
> > As Richard said, npm and bower are not competitors. You use npm to
> > install bower, and you use bower to download Angular, jQuery, Bootstrap
> > and other static files. These are the static files that you will want to
> > include when you finally deploy the web app to your server.
> >
> > Before using Bower, people would simply download Angular from the
> > projects homepage and check it into version control. Bower is not doing
> > much, but using it avoids this bad practice.
> >
> > There is often a kind of "compilation step" between bower downloading a
> > dependency and the deployment on the webserver: minification and
> > compression of the JavaScript and CSS. Concatenating and minifying the
> > files serve to reduce the number of HTTP requests -- which can make an
> > app much faster.
> >
> > Finally, you use Grunt/Gulp to execute other tools during development.
> > These tools could be a local web server, it could be running the unit
> > tests. Grunt is only a convenience tool here -- think of it as a kind of
> > Makefile that tells you how to lunch various tasks.
>
> Thank you for your explanations.
>
> The way I see it, we would need:
> - Grunt both in the development environment and packaged (to run tests,
> etc.),

I'm increasingly coming to think that we can avoid grunt.

- serve is handled by django runserver
- test running is handled by tox
- livereload could be handled by https://pypi.python.org/pypi/livereload
(it'd be really nice if someone could get support for this rolled in to
django-livereload...)
- watch is not handled by anything and it would be a shame to miss out on
automatic linting/testing, but I think we can live without it

So, the tools we need packaged for Linux are:

- karma
- jasmine (already in Fedora, I believe)
- selenium (I believe this is already done in Fedora and Debian)
- phantomjs (definitely already done!)


> - Bower in the development environment,
> - Bower configuration file in two copies, one for global-requirements,
> and one for the Horizon's local requirements. Plus a gate job that makes
> sure no new library or version gets included in the Horizon's before
> getting into the global-requirements,

Could you perhaps elaborate on this? How do you see the workflow working
here?

Given that Horizon already integrates with xstatic, it would be messy (and
potentially confusing) to include something so it *also* integrated with
bower. I was envisaging us creating a tool which generates xstatic packages
from bower packages. I'm not the first to think along these lines
http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html

I will be looking into creating such a tool today.


> - A tool, probably a script, that would help packaging the Bower
> packages into DEB/RPM packages. I suspect the Debian/Fedora packagers
> already have a semi-automatic solution for that.

Yep, that is indeed their problem that they'd have already solved for the
existing xstatic packages.


> - A script that would generate a file with all the paths to those
> packaged libraries, that would get included in Horizon's settings.py

If we stick with xstatic, we don't need this :)


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 18 November 2014 10:59, Richard Jones  wrote:

> On 17 November 2014 21:54, Radomir Dopieralski 
> wrote:
> > - Bower in the development environment,
> > - Bower configuration file in two copies, one for global-requirements,
> > and one for the Horizon's local requirements. Plus a gate job that makes
> > sure no new library or version gets included in the Horizon's before
> > getting into the global-requirements,
>
> Could you perhaps elaborate on this? How do you see the workflow working
> here?
>
> Given that Horizon already integrates with xstatic, it would be messy (and
> potentially confusing) to include something so it *also* integrated with
> bower. I was envisaging us creating a tool which generates xstatic packages
> from bower packages. I'm not the first to think along these lines
> http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html
>
> I will be looking into creating such a tool today.
>

I wrote the tool today, and you can find it here:

https://github.com/r1chardj0n3s/flaming-shame

(github auto-named the repository for me - it's like it KNOWS)

I've proposed to Thomas Waldmann that this be included in the xstatic
package.

It doesn't handle dependencies at all - I'm not sure it should or could
sensibly.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 18 November 2014 19:22, Radomir Dopieralski 
wrote:

> On 18/11/14 00:59, Richard Jones wrote:
> > On 17 November 2014 21:54, Radomir Dopieralski  > <mailto:openst...@sheep.art.pl>> wrote:
> >> - Bower in the development environment,
> >> - Bower configuration file in two copies, one for global-requirements,
> >> and one for the Horizon's local requirements. Plus a gate job that makes
> >> sure no new library or version gets included in the Horizon's before
> >> getting into the global-requirements,
> >
> > Could you perhaps elaborate on this? How do you see the workflow working
> > here?
>
> Basically we would have an additional file in the global-requirements
> directory, for listing the JavaScript dependencies. Then we would have a
> check on the Horizon's gate that would check Horizon's Bower
> configuration against that global-requirements file.
>
> This way we keep the same process for JavaScript libraries as we have
> for Python libraries: first you submit a patch to the
> global-requirements and have the dependency accepted by the infra team
> and packagers (checked against licenses, version conflicts, etc.), and
> then you add it to Horizon's dependencies. Of course you can submit both
> patches at once, just the Horizon one will fail the gate until the
> global-requirements one gets merged.
>
> > Given that Horizon already integrates with xstatic, it would be messy
> > (and potentially confusing) to include something so it *also* integrated
> > with bower. I was envisaging us creating a tool which generates xstatic
> > packages from bower packages. I'm not the first to think along these
> > lines
> http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html
>
> If we use Bower, we don't need to use Xstatic. It would be pure
> overhead. Bower already takes care of tracking releases and versions,
> and of bundling the files. All we need is a simple line in the
> settings.py telling Django where it puts all the files -- we don't
> really need Xstatic just for that. The packagers can then take those
> Bower packages and turn them into system packages, and just add/change
> the paths in settings.py to where they put the files. All in one place.
>

I guess I got the message that turning bower packages into system packages
was something that the Linux packagers were not keen on. Did I get the
message wrong there? If so, *and* we can get the bower stuff through #infra
and global-requirements then yes, we should totally try to avoid adding the
xstatic layer :)


Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] pypi-mirror is now unsupported - what do we do now?

[Richard Jones]

On 10 July 2014 02:19, Ben Nemec  wrote:

> On 07/08/2014 11:05 PM, Joe Gordon wrote:
> > On Tue, Jul 8, 2014 at 8:54 PM, James Polley  wrote:
> >
> >> It may not have been clear from the below email, but clarkb clarifies on
> >> https://bugs.launchpad.net/openstack-ci/+bug/1294381 that the infra
> team
> >> is no longer maintaining pypi-mirror
> >>
> >> This has been a very useful tool for tripleo. It's much simpler for new
> >> developers to set up and use than a full bandersnatch mirror (and
> requires
> >> less disk space), and it can create a local cache of wheels which saves
> >> build time.
> >>
> >> But it's now unsupported.
> >>
> >> To me it seems like we have two options:
> >>
> >> A) Deprecate usage of pypi-mirror; update docs to instruct new devs in
> >> setting up a local bandersnatch mirror instead
> >> or
> >> B) Take on care-and-feeding of the tool.
> >> or, I guess,
> >> C) Continue to recommend people use an unsupported unmaintained
> >> known-buggy tool (it works reasonably well for us today, but it's going
> to
> >> work less and less well as time goes by)
> >>
> >> Are there other options I haven't thought of?
> >>
> >
> > I don't know if this fits your requirements but I use
> > http://doc.devpi.net/latest/quickstart-pypimirror.html for my
> development
> > needs.
>
> Will that also cache wheels?  In my experience, wheels are one of the
> big time savers in tripleo so I would consider it an important feature
> to maintain, however we decide to proceed.
>

Yes, devpi caches wheels.

I would suggest that if the pip cache approach isn't appropriate then devpi
probably a good solution (though I don't know your full requirements).

The big difference between using devpi and pip caching would be that devpi
will allow you to install packages when you're offline.


   Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] pypi-mirror is now unsupported - what do we do now?

[Richard Jones]

It'd be more accurate to say that it doesn't support automatic generation
of wheels - how would it know what to generate?

The wheels must be created by a build system. The devpi client has support
for building and uploading wheels, and devpi is designed to support having
locally-built wheels supplement existing packages. I'd be happy to
elaborate.


 Richard


On 10 July 2014 10:43, Donald Stufft  wrote:

>
> On Jul 9, 2014, at 7:07 PM, Richard Jones  wrote:
>
> On 10 July 2014 02:19, Ben Nemec  wrote:
>
>> On 07/08/2014 11:05 PM, Joe Gordon wrote:
>> > On Tue, Jul 8, 2014 at 8:54 PM, James Polley 
>> wrote:
>> >
>> >> It may not have been clear from the below email, but clarkb clarifies
>> on
>> >> https://bugs.launchpad.net/openstack-ci/+bug/1294381 that the infra
>> team
>> >> is no longer maintaining pypi-mirror
>> >>
>> >> This has been a very useful tool for tripleo. It's much simpler for new
>> >> developers to set up and use than a full bandersnatch mirror (and
>> requires
>> >> less disk space), and it can create a local cache of wheels which saves
>> >> build time.
>> >>
>> >> But it's now unsupported.
>> >>
>> >> To me it seems like we have two options:
>> >>
>> >> A) Deprecate usage of pypi-mirror; update docs to instruct new devs in
>> >> setting up a local bandersnatch mirror instead
>> >> or
>> >> B) Take on care-and-feeding of the tool.
>> >> or, I guess,
>> >> C) Continue to recommend people use an unsupported unmaintained
>> >> known-buggy tool (it works reasonably well for us today, but it's
>> going to
>> >> work less and less well as time goes by)
>> >>
>> >> Are there other options I haven't thought of?
>> >>
>> >
>> > I don't know if this fits your requirements but I use
>> > http://doc.devpi.net/latest/quickstart-pypimirror.html for my
>> development
>> > needs.
>>
>> Will that also cache wheels?  In my experience, wheels are one of the
>> big time savers in tripleo so I would consider it an important feature
>> to maintain, however we decide to proceed.
>>
>
> Yes, devpi caches wheels.
>
> I would suggest that if the pip cache approach isn't appropriate then
> devpi probably a good solution (though I don't know your full requirements).
>
> The big difference between using devpi and pip caching would be that devpi
> will allow you to install packages when you're offline.
>
>
>Richard
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> It doesn’t generate Wheels though, it’ll only cache them if they exist on
> PyPI already.
>
> -
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] pypi-mirror is now unsupported - what do we do now?

[Richard Jones]

On 10 July 2014 23:27, Mulcahy, Stephen  wrote:
> When I last tested bandersnatch, it didn’t work well behind a proxy (in
fact most of the existing pypi mirroring tools suffered from the same
problem) – pypi-mirror has worked extremely well for mirroring a subset of
pypi and doing so behind a proxy. I’d also echo the requirement for a tool
that provides wheels as we have seen significant performance improvement
from using wheels with TripleO

devpi works behind a proxy. If bandersnatch doesn't then that bug should be
addressed ASAP. I'm in contact with its author regarding that.

I'm currently investigating a sensible approach to having wheels be
automatically built (for the most sensible value of "automatic" that we can
determine ).


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] unit test improvements: work with DOM, not raw HTML strings?

[Richard Jones]

Hi all,

I've noticed that we have quite a few places in our unit test suite that
test for correctness of behaviour by searching for quite complex HTML
fragments in page renders. An example would be
openstack_dashboard/dashboards/project/access_and_security/floating_ips/tests.py

(there's plenty more - search for "expected_string")

The code in question is testing far more than it should. We are testing for
the presence of the "disabled" class in an element which has a clear id
attribute by constructing the whole element HTML, including label and title
text, glyph icon, other classes, the tag type itself, etc. All of those
things are quite irrelevant to the test in question, but any change to
those will cause the test to break unnecessarily.

In these cases a far more robust and targeted unit test would construct a
DOM from the rendered HTML and use semantic searches to determine the
correctness of behaviour. I would recommend we consider using something
like https://django-with-asserts.rtfd.org/ which allow us to write:

 with self.assertHTML(res, element_id='security_groups__action_create') as
elem:
self.assertContains(elem.attrib['class'], 'disabled')

Who's with me?


Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Pagination in thre API

[Richard Jones]

As a consumer of such APIs on the Horizon side, I'm all for consistency in
pagination, and more of it, so yes please!

On 5 November 2015 at 13:24, Tony Breeds  wrote:

> On Thu, Nov 05, 2015 at 01:09:36PM +1100, Tony Breeds wrote:
> > Hi All,
> > Around the middle of October a spec [1] was uploaded to add
> pagination
> > support to the os-hypervisors API.  While I recognize the use case it
> seemed
> > like adding another pagination implementation wasn't an awesome idea.
> >
> > Today I see 3 more requests to add pagination to APIs [2]
> >
> > Perhaps I'm over thinking it but should we do something more strategic
> rather
> > than scattering "add pagination here".
> >
> > It looks to me like we have at least 3 parties interested in this.
> >
> > Yours Tony.
> >
> > [1] https://review.openstack.org/#/c/234038
> > [2]
> https://review.openstack.org/#/q/message:pagination+project:openstack/nova-specs+status:open,n,z
>
> Sorry about the send without complete subject.
>
> Yours Tony.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][policy] Exposing hypervisor details to users

[Richard Jones]

+1 this would definitely be the best approach from Horizon's perspective
since we can cache the capabilities per-flavour. Having to request
capabilities per-instance would be an unreasonable burden on the poor users
of Horizon.

On 6 November 2015 at 23:09, Sean Dague  wrote:

> On 11/06/2015 04:49 AM, Daniel P. Berrange wrote:
> > On Fri, Nov 06, 2015 at 05:08:59PM +1100, Tony Breeds wrote:
> >> Hello all,
> >> I came across [1] which is notionally an ironic bug in that horizon
> presents
> >> VM operations (like suspend) to users.  Clearly these options don't
> make sense
> >> to ironic which can be confusing.
> >>
> >> There is a horizon fix that just disables migrate/suspened and other
> functaions
> >> if the operator sets a flag say ironic is present.  Clealy this is sub
> optimal
> >> for a mixed hv environment.
> >>
> >> The data needed (hpervisor type) is currently avilable only to admins,
> a quick
> >> hack to remove this policy restriction is functional.
> >>
> >> There are a few ways to solve this.
> >>
> >>  1. Change the default from "rule:admin_api" to "" (for
> >> os_compute_api:os-extended-server-attributes and
> >> os_compute_api:os-hypervisors), and set a list of values we're
> >> comfortbale exposing the user (hypervisor_type and
> >> hypervisor_hostname).  So a user can get the hypervisor_name as
> part of
> >> the instance deatils and get the hypervisor_type from the
> >> os-hypervisors.  This would work for horizon but increases the API
> load
> >> on nova and kinda implies that horizon would have to cache the data
> and
> >> open-code assumptions that hypervisor_type can/can't do action $x
> >>
> >>  2. Include the hypervisor_type with the instance data.  This would
> place the
> >> burdon on nova.  It makes the looking up instance details slightly
> more
> >> complex but doesn't result in additional API queries, nor caching
> >> overhead in horizon.  This has the same opencoding issues as Option
> 1.
> >>
> >>  3. Define a service user and have horizon look up the hypervisors
> details via
> >> that role.  Has all the drawbacks as option 1 and I'm struggling to
> >> think of many benefits.
> >>
> >>  4. Create a capabilitioes API of some description, that can be queried
> so that
> >> consumers (horizon) can known
> >>
> >>  5. Some other way for users to know what kind of hypervisor they're
> on, Perhaps
> >> there is an established image property that would work here?
> >>
> >> If we're okay with exposing the hypervisor_type to users, then #2 is
> pretty
> >> quick and easy, and could be done in Mitaka.  Option 4 is probably the
> best
> >> long term solution but I think is best done in 'N' as it needs lots of
> >> discussion.
> >
> > I think that exposing hypervisor_type is very much the *wrong* approach
> > to this problem. The set of allowed actions varies based on much more
> than
> > just the hypervisor_type. The hypervisor version may affect it, as may
> > the hypervisor architecture, and even the version of Nova. If horizon
> > restricted its actions based on hypevisor_type alone, then it is going
> > to inevitably prevent the user from performing otherwise valid actions
> > in a number of scenarios.
> >
> > IMHO, a capabilities based approach is the only viable solution to
> > this kind of problem.
>
> Right, we just had a super long conversation about this in #openstack-qa
> yesterday with mordred, jroll, and deva around what it's going to take
> to get upgrade tests passing with ironic.
>
> Capabilities is the right approach, because it means we're future
> proofing our interface by telling users what they can do, not some
> arbitrary string that they need to cary around a separate library to
> figure those things out.
>
> It seems like capabilities need to exist on flavor, and by proxy instance.
>
> GET /flavors/bm.large/capabilities
>
> {
>  "actions": {
>  'pause': False,
>  'unpause': False,
>  'rebuild': True
>  ..
>   }
>
> A starting point would definitely be the set of actions that you can
> send to the flavor/instance. There may be features beyond that we'd like
> to classify as capabilities, but actions would be a very concrete and
> attainable starting point. With microversions we don't have to solve
> this all at once, start with a concrete thing and move forward.
>
> Sending an action that was "False" for the instance/flavor would return
> a 400 BadRequest high up at the API level, much like input validation
> via jsonschema.
>
> This is nothing new, we've talked about it in the abstract in the Nova
> space for a while. We've yet had anyone really take this on. If you
> wanted to run with a spec and code, it would be welcome.
>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-

Re: [openstack-dev] [Horizon] Bug day! Yeah!

[Richard Jones]

Let's do it. I'd like to suggest we use an etherpad to keep track of what
people have done. If it's not created when I start my day, I'll make one.


 Richard

On 19 November 2015 at 22:19, Rob Cresswell (rcresswe) 
wrote:

> Hey folks,
>
> Our bug list is… rather large. We’ve discussed having a bug day, where as
> a community we all dedicate some time to triaging bugs and discussing in
> the IRC channel as we go.
>
> First off, see the docs about bug triage:
> https://wiki.openstack.org/wiki/BugTriage
>
> Secondly, lets pick a date. I’d suggest next Tuesday (24th November); if
> that doesn’t work for a good number of us, then the following Tuesday (1st
> December). Trying to account for Thanksgiving :)
>
> Rob
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Proposal to add Timur Sufiev to horizon-core

[Richard Jones]

+1!

On 3 December 2015 at 07:31, Matthias Runge  wrote:

> On 02/12/15 19:52, David Lyle wrote:
> > I propose adding Timur Sufiev[1] to horizon-core.
> >
> > Over the last several cycles Timur has consistently been providing
> > great reviews, actively participating in the Horizon community, and
> > making meaningful contributions particularly around testing and
> > stability.
> >
> > Please respond with comments, +1s, or objections within one week.
> >
> Oh, yes please! Timur has been doing a great job, not only over the past
> cycle.
>
> Matthias
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon][release][infra] New xstatic package release proposal

[Richard Jones]

TL;DR: "Releasing xstatic packages" https://review.openstack.org/#/c/253296/

xstatic  is an existing standard for packaging up
static resources for web development (HTML, Javascript and CSS) as Python
modules. Horizon makes use of two dozen xstatic packages.

We need a better-organised method of releasing those packages. Currently
those packages are maintained and released in an ad-hoc manner by
individuals who come and go in the community, with differing release
processes, requiring maintenance handovers, sometimes with people who go
quiet. This does not scale to the dozens of xstatic packages Horizon wishes
to consume.

We therefore wish to use OpenStack's existing release pipeline, handing
over upload to PyPI to that system, while also allowing review of updates
to the packages.

A problem arises in doing so: xstatic packages version themselves using the
three-digit version of the static package they bundle (eg. JQuery-1.2.3)
with an added build number (eg. xstatic-jquery-1.2.3.4) to allow
re-building if the packaging goes wrong. Currently we cannot release these
packages through the OpenStack release pipeline because pbr (a requirement
of that pipeline) does not allow 4 digit version strings. A proposal to
extend pbr to allow 4 digit versions was abandoned today:
https://review.openstack.org/#/c/205623/ (in short, we are only using the
version git tag extraction facility of pbr, and even then not all of that,
and pbr would become more complex with the special-casing required to ...
special-case xstatic packaging).

We will look to use setuptools_scm instead of pbr. Some support work will
be required to make the proposal above happen. This is outlined in
"Releasing xstatic packages" https://review.openstack.org/#/c/253296/

Also, a patch has been submitted to a sample xstatic package implementing
the proposed new workflow: https://review.openstack.org/#/c/252752/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Proposal to add Richard Jones tohorizon-core

[Richard Jones]

Thank you all!

On 9 December 2015 at 04:23, David Lyle  wrote:

> Since the results are unanimous, closing early. Welcome Richard!
>
> David
>
> On Thu, Dec 3, 2015 at 12:08 PM, Thai Q Tran  wrote:
> > An equally BIG +1 from me! Thanks for all the reviews and patches from
> your
> > minions Richard!
> >
> >
> > - Original message -
> > From: David Lyle 
> > To: OpenStack Development Mailing List <
> openstack-dev@lists.openstack.org>
> > Cc:
> > Subject: [openstack-dev] [horizon] Proposal to add Richard Jones to
> > horizon-core
> > Date: Wed, Dec 2, 2015 10:57 AM
> >
> > I propose adding Richard Jones[1] to horizon-core.
> >
> > Over the last several cycles Timur has consistently been providing
> > great reviews, actively participating in the Horizon community, and
> > making meaningful contributions around angularJS and overall project
> > stability and health.
> >
> > Please respond with comments, +1s, or objections within one week.
> >
> > Thanks,
> > David
> >
> > [1]
> >
> http://stackalytics.com/?module=horizon-group&user_id=r1chardj0n3s&release=all
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] New procedure for releasing xstatic packages

[Richard Jones]

Hi folks,

It's been a lng time coming, but we're almost there. I've put up a BP
outlining the steps we need to get over the line and start releasing those
looong awaited updates of our xstatic packages:

https://blueprints.launchpad.net/horizon/+spec/xstatic-release-process

I'll be working with Tony Breeds and Joshua Hesketh over the next few days
to work through the remaining bits of that BP. Please feel free to review
the BP and the linked changes - in particular the documentation change
which has more detail than the BP:

https://review.openstack.org/#/c/289142

One of the potential gotchas is that we're enforcing that the git tag
version string matches the xstatic package's BUILD version string as part
of the release process (as far as I can tell this is a first for an
OpenStack release process). The downside of this is that you won't know if
it's been rejected unless you sign up to a particular mailing list:

http://lists.openstack.org/cgi-bin/mailman/listinfo/release-job-failures

Well, eventually you'll give up on seeing the package appear in pypi, but
that list is slightly more immediate.


Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] xstatic publishing update - broken xstatic-angular-bootstrap

[Richard Jones]

Hi folks,

I released a couple of broken xstatic-angular-bootstrap packages to pypi
yesterday - 0.11.0.4 and 0.11.0.5. There was to be an automated release of
0.11.0.6 through openstack-release, but I've abandoned that as it contained
the same breakage. I've performed an emergency direct upload of 0.11.0.7 to
pypi which fixes the issue, and will propose an automated release of
0.11.0.8 some time this weekend.

Future packages should not run into this sort of issue - the root cause is
a bug in the xstatic-release script, which I'm also going to address and
release as 1.1.2 this weekend.


Sorry for the trouble,

  Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] naming of Javascript bits

[Richard Jones]

Yep, thanks folks. Apart from the existing styleguide arguing with my
preferred approach, it seems like we agree on something that should be
pretty consistent. Thanks for putting up that consistency patch Rajat.

On 8 January 2016 at 06:49, Rajat Vig  wrote:

> I created a patch based off what Richard used as an example to highlight
> inconsistencies and made it consistent.
>
> https://review.openstack.org/#/c/264942/
>
> Have a look and if it feels fine, we can start changing what else exists
> on similar lines.
> Suggestions welcome.
>
> -Rajat
>
> On Thu, Jan 7, 2016 at 10:14 AM Thai Q Tran  wrote:
>
>> I 2nd that, should rename the file to delete.service.js to match service
>> name.
>>
>>
>> - Original message -
>> From: Rajat Vig 
>> To: "OpenStack Development Mailing List (not for usage questions)" <
>> openstack-dev@lists.openstack.org>
>> Cc:
>> Subject: Re: [openstack-dev] [Horizon] naming of Javascript bits
>> Date: Thu, Jan 7, 2016 12:03 AM
>>
>> Richard
>>
>> My preference is the same as what you've got there.
>> Fully namespaced Services and Controller allow for better reusability and
>> possibly maintainability.
>> If all "deleteService" were named just that, it'll be mighty confusing to
>> use it in other places.
>>
>> With regards to tying the folder path and the Service/Controller I'd
>> mostly go with that as that encourages simpler rules on how to namespace.
>>
>> For the particular patch you mentioned, the namespaces had a bit of churn
>> which is sort of reflected in what exists in the patch now.
>>
>> If we decide a convention, then we can go and change the bits when the
>> files change next.
>>
>> -Rajat
>>
>>
>>
>> On Wed, Jan 6, 2016 at 10:30 PM Richard Jones 
>> wrote:
>>
>> Hi Horizon folks,
>>
>> We've been pretty good about namespacing the new angular code (to the
>> extreme of having a bunch of very similar module files littered around, but
>> that's angular/JS for you, so I'm not going to go on about it ).
>>
>> Anyhoo. One thing I've noticed is that the services, factories and
>> controllers inside those modules aren't being consistently named. We have
>> got a mix of:
>>
>> Launch Instance:
>>
>>   .module('horizon.dashboard.project.workflow.launch-instance')
>>   .factory('launchInstanceModel', launchInstanceModel);
>>
>> The new Images panel:
>>
>>   .module('horizon.app.core.images')
>>   .factory('horizon.app.core.images.row-actions.service', rowActions);
>>
>> and in the same patch:
>>
>>   .module('horizon.app.core.images')
>>   .factory('horizon.app.core.images.actions.deleteService',
>> deleteService);
>>
>> I actually prefer the second form because it matches the filename
>> ("row-actions.service.js") even though the module namespace doesn't match
>> the file path ("/static/app/core/images/table/").
>>
>> Your thoughts?
>>
>>
>>  Richard
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Updating XStatic packages

[Richard Jones]

In terms of process, obviously updating the data in the xstatic is
relatively easy (for most packages) but I wonder from a review standpoint
what sort of process we should have here. To approve a merge of an xstatic
package update, I feel like I should have, in order of preference:

1. a link to the change notes for the project in question so I can see what
the impact of the update is, or
2. a list of those changes in the bug or change message for the xstatic
update (i.e. the relevant information from #1), and
3. an explanation of what existing problems are being solved in Horizon
through the update (if any) or what impact there will be on Horizon with
the upgrade (i.e. an examination of #2 in the context of Horizon).

I know that we often need to justify package updates to downstream
packagers, and having this information will certainly help our case. Or is
this all a bit too much burden?


 Richard

On 7 January 2016 at 04:36, Rajat Vig  wrote:

> I did update some of the low hanging packages where the upgrades seemed
> safe to do and noted the patch numbers in the same EtherPad.
>
> I wasn't sure what to mark the effort against. So I created some bugs.
> Should it be a blueprint?
>
> -Rajat
>
> On Wed, Jan 6, 2016 at 2:18 AM, Rob Cresswell (rcresswe) <
> rcres...@cisco.com> wrote:
>
>> Hi all,
>>
>> While the automated system is broken, I’d like to work on manually
>> releasing a few of the XStatic packages. This will *only* be the release
>> stage; we will still use gerrit to review the package content as usual.
>>
>> List of packages current versions, and their upstreams, can be found
>> here: https://etherpad.openstack.org/p/horizon-libs
>>
>> If anyone has spare time, please consider investigating some of the
>> dependencies listed above. To update an XStatic package, propose a change
>> to its repo as you would with Horizon; they are all in the OpenStack
>> namespace. For example, Xstatic-Angular can be found at
>> http://git.openstack.org/cgit/openstack/xstatic-angular/
>>
>> Thanks,
>> Rob
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Routing in Horizon

[Richard Jones]

The  tag addition has had to be reverted as it broke other parts of
the application (notably lazy loaded tabs like Instance Details), sadly.

Regarding which router to use - I've used the built-in router in the past
quite successfully. I think I'd want to see a solid reason for using a 3rd
party one. It could be that nested routes are part of that, but I kinda see
that as a convenience thing, unless I'm missing something core to how
routing is planned to be done. Then again we really haven't had any
discussion about how we see routing work. The one patch that's up that uses
routing seems perfectly able to do so without needing extended capabilities
of 3rd party routers.


 Richard


On 12 January 2016 at 09:44, Tripp, Travis S  wrote:

> Rajat,
>
> Thanks for starting some discussion here.  I think your target=“_self” is
> taken care of now, right?
>
> Re: ng-route vs ui-router - everything I have ever seen or been told
> supports using ui-router instead of ng-route.  I know a quick google search
> supports that, but let me google that for all of us and give several
> references:
>
> http://www.funnyant.com/angularjs-ui-router/
> http://www.amasik.com/angularjs-ngroute-vs-ui-router/
>
> http://stackoverflow.com/questions/21023763/angularjs-difference-between-angular-route-and-angular-ui-router
>
> http://www.pearpages.com/javascript/angularjs/routing/2015/10/13/ngroute-vs-ui-router.html
>
> http://stackoverflow.com/questions/32523512/ui-router-vs-ngroute-for-sinlge-page-app
>
> So, I’m wondering if there’d been any discussion I missed on why not bring
> in ui-router?
>
> Of course there is question using the new router in angular vs ui-router,
> but finding many pros- cons- on that seems to be a bit more difficult.
> Since it is 1.5 / 2.0 and neither are past rc / beta, it doesn’t seem like
> something we can debate too well.
>
> https://angular.github.io/router/getting-started
>
> http://www.angulardaily.com/2015/12/angularintroduction-to-ngnewrouter-vs.html
>
> Thanks,
> Travis
>
> From: Rajat Vig mailto:raj...@thoughtworks.com>>
> Reply-To: OpenStack List  openstack-dev@lists.openstack.org>>
> Date: Thursday, January 7, 2016 at 1:53 AM
> To: OpenStack List  openstack-dev@lists.openstack.org>>
> Subject: [openstack-dev] [Horizon] Routing in Horizon
>
> Hi Everyone
>
> One of my recent patches which enabled HTML5 based routing via Angular
> merged, some interesting things spun out.
> I'd to scramble a few patches to get things
> ​​ back the same way
> for all new Angular Panels.
>
> I also realized that getting Horizon to an SPA with Angular requires more
> thought than mere fixing the current burning issue.
> This mail's intent is to spur a direction on how we do routing in Angular
> and how do Angular Panels go back/forth between themselves and older Django
> panels.
>
> The patch https://review.openstack.org/#/c/173885/ is possibly the first
> of many to use Angular based routing.
> It currently uses ngRoute as the library was included in the
> xstatic-angular package.
>
> What I'm roughly thinking to solve some of the immediate issues (there's
> possbily much more that I'm not)
>
> 1. If we are going to go with the SPA route, then all Panels need to
> indicate that they are Angular based.
> For panels that are Angular, they need to declare routes they'd like to
> manage.
>
> 2. All tags on Angular Panels (including header, sidebar, footer) which
> don't route to Angular Panels will need the attribute target="_self" else
> Angular will not allow navigation to those links.
>
> All sidebar links currently have the attribute set but headers and footers
> don't.
> Sidebar links for Angular shouldn't have the attribute else SPA like
> behavior will not happen.
> This will need to be documen
> ​tation​
> ​
> ​
> ​
> .
>
> 3. That implies yet another problem with the Spinner Modal which gets
> activated on all sidebar clicks.
> It'll need to be done differently for Angular routing vs hrefs still with
> Django.
> The current spinner relies on a browser page refresh to disappear.
>
> Then there's ngRoute.
> Routing needs in Angular may need to handle route conflicts and maybe
> nested routes.
> There are other, possibly better options that we could consider
> 1. https://github.com/angular-ui/ui-router
> 2. https://angular.github.io/router/
>
> I've been part of the community for not long enough yet and I'm not yet
> completely aware of what exists outside the Horizon codebase so I might be
> missing somethings as well.
>
> Regards
> Rajat
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org

[openstack-dev] [Horizon] Handling 401 in new REST API

[Richard Jones]

Hi fellow angular/REST Horizon developers,

I'd like to propose that we handle HTTP 401 responses at the core of the
new angular code interfacing to our new REST API so that *any* 401 just
does basically what the Django code used to: redirect to the login page
with a "from".

What do y'all think?


Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Handling 401 in new REST API

[Richard Jones]

Hmm. That client-side redirect isn't working consistently then. More
investigation required...

On 29 January 2016 at 05:04, Thai Q Tran  wrote:

> I am assuming that you meant redirecting on the server side. We already
> have a similar one in place on the client side.
> https://github.com/openstack/horizon/blob/master/horizon/static/framework/framework.module.js#L65
>
> Currently, you're not able to hit the REST layer directly anyway, You get
> a "request must be ajax" message. You have to login and then use a REST
> client if you want data. So here's my noob question of the day, what are we
> gaining in addition if we redirect on the server side? Totally not meant to
> be a show stopper, just wondering what the benefits are.
>
>
> - Original message -
> From: David Lyle 
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Cc:
> Subject: Re: [openstack-dev] [Horizon] Handling 401 in new REST API
> Date: Thu, Jan 28, 2016 8:46 AM
>
> I think that's the sane thing to do.
>
> David
>
> On Thu, Jan 28, 2016 at 2:55 AM, Richard Jones 
> wrote:
> > Hi fellow angular/REST Horizon developers,
> >
> > I'd like to propose that we handle HTTP 401 responses at the core of the
> new
> > angular code interfacing to our new REST API so that *any* 401 just does
> > basically what the Django code used to: redirect to the login page with a
> > "from".
> >
> > What do y'all think?
> >
> >
> > Richard
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Recent integration tests failures

[Richard Jones]

Ugh, dependencies with breaking API changes in minor point releases :/

On 2 February 2016 at 04:53, Timur Sufiev  wrote:

> Maintainers of outside dependencies continue to break our stuff :(. New
> issue is https://bugs.launchpad.net/horizon/+bug/1540495 patch is
> currently being checked by Jenkins
>
> On Sat, Jan 30, 2016 at 2:28 PM Timur Sufiev  wrote:
>
>> Problematic Selenium versions have been successfully excluded from
>> Horizon test-requirements, if you still experiencing the error described
>> above, rebase your patch onto the latest master.
>> On Fri, 29 Jan 2016 at 12:36, Itxaka Serrano Garcia 
>> wrote:
>>
>>> Can confirm, had the same issue locally, was fixed after a downgrade to
>>> selenium 2.48.
>>>
>>>
>>> Good catch!
>>>
>>> Itxaka
>>>
>>> On 01/28/2016 10:08 PM, Timur Sufiev wrote:
>>> > According to the results at
>>> > https://review.openstack.org/#/c/273697/1 capping Selenium to be not
>>> > greater than 2.49 fixes broken tests. Patch to global-requirements is
>>> > here: https://review.openstack.org/#/c/273750/
>>> >
>>> > On Thu, Jan 28, 2016 at 9:22 PM Timur Sufiev >> > > wrote:
>>> >
>>> > Hello, Horizoneers
>>> >
>>> > You may have noticed recent integration tests failures seemingly
>>> > unrelated to you patches, with a stacktrace like:
>>> > http://paste2.org/2Hk9138U I've already filed a bug for that,
>>> > https://bugs.launchpad.net/horizon/+bug/1539197 Appears to be a
>>> > Selenium issue, currently investigating it.
>>> >
>>> >
>>> >
>>> >
>>> >
>>> __
>>> > OpenStack Development Mailing List (not for usage questions)
>>> > Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [javascript] [qa] [eslint-config-openstack] Nominating Cindy Lu for eslint-config-openstack-core

[Richard Jones]

+1

On 10 July 2015 at 05:53, David Lyle  wrote:

> On Thu, Jul 9, 2015 at 1:33 PM, Michael Krotscheck 
> wrote:
>
>> Hey everyone!
>>
>> We've been hard at work getting some solid, shared code style rules in
>> place for our javascript code, and nobody's been more diligent about this
>> than Cindy Lu. Since we've managed to encapsulate our eslint configuration
>> into a separate project (much like hacking), I'd like to nominate Cindy Lu
>> as a core!
>>
>> This will give her the power to tell you whether spaces or tabs are
>> better. So be warned :D
>>
>> Michael
>>
>>
> +1
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] URL Sanity

[Richard Jones]

Interesting idea, and in general I'm for consistency. I can't speak
directly to the network/port question, though it seems to me that if ports
must be attached to networks then it makes sense for the URL to reflect
that.

On the other hand, some could argue that the django URL routing is ...
legacy ... and shouldn't me messed with :)

On the gripping hand, thinking about this could inform future angular
routing planning...

On 2 September 2015 at 00:39, Rob Cresswell (rcresswe) 
wrote:

> Hi all,
>
> I recently started looking into properly implementing breadcrumbs to make
> navigation clearer, especially around nested resources (Subnets Detail
> page, for example). The idea is to use the request.path to form a logical
> breadcrumb that isn’t dependent on browser history (
> https://review.openstack.org/#/c/129985/3/horizon/browsers/breadcrumb.py ).
> Unfortunately, this breaks down quite quickly because we use odd patterns
> like `//detail`, and `/`
> doesn’t exist.
>
> This made me realise how much of an inconsistent mess the URL patterns
> are.  I’ve started cleaning them up, so we move from these patterns:
>
> `/admin/networks//detail` - Detail page for a Network
> `/admin/networks//addsubnet` - Create page for a Subnet
>
> To patterns in line with usual CRUD usages, such as:
>
> `/admin/networks/` - Detail page for a Network
> `/admin/networks//subnets/create` - Create page for a Subnet
>
> This is mostly trivial, just removing extraneous words and adding
> consistency, with end goal being every panel following patterns like:
>
> `/` - Index page
> `//` - Detail page for a single resource
> `//create` - Create new resource
> `///update` - Update a resource
>
> This gets a little complex around nested items. Should a Port for example,
> which has a unique ID, be reachable in Horizon by just its ID? Ports must
> always be attached to a network as I understand it. There are multiple ways
> to express this:
>
> `/networks/ports/` - Current implementation
> `/networks//ports/` - My preferred structure
> `/ports/` - Another option
>
> Does anyone have any opinions on how to handle this structuring, or if
> it’s even necessary?
>
> Regards,
> Rob
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] python-selenium landed in Debian main today (in Debian Experimental for the moment)

[Richard Jones]

On 9 September 2015 at 05:35, Thomas Goirand  wrote:

> After the non-free files were removed from the package (after I asked
> for it through the Debian bug https://bugs.debian.org/770232), Selenium
> was uploaded and reached Debian Experimental in main today (ie: Selenium
> is not in non-free section of Debian anymore). \o/
>

\o/


Now, I wonder: can the Horizon team use python-selenium as uploaded to
> Debian experimental today? Can we run the Selenium unit tests, even
> without the browser plugins? It is my understanding that it's possible,
> if we use something like PhantomJS, which is also available in Debian.
>

We can't use PhantomJS as a webdriver as a couple of the tests interact
with file inputs and ghostdriver doesn't support those, sadly (and the
developer of ghostdriver is MIA). We are pretty much stuck with just
Firefox as the webdriver.


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Horizon Productivity Suggestion

[Richard Jones]

The etherpad that we had running for the bulk of L was really handy;
something like that would be great to keep using to let folks know what is
in play.

On 29 September 2015 at 19:32, Rob Cresswell (rcresswe) 
wrote:

> I wasn’t really envisioning a big discussion on the bugs; more like a
> brief notice period to let reviewers know high-priority items. Could
> definitely spend longer over it if that is preferred. Timing aside, the
> overall idea sounds good though?
>
> Lin: That’s a good idea. A wiki page would probably suffice.
>
> Rob
>
> From: Lin Hua Cheng 
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Date: Tuesday, 29 September 2015 04:11
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Subject: Re: [openstack-dev] [Horizon] Horizon Productivity Suggestion
>
> I agree with Travis that 2-3 minutes is not enough, that may not be even
> enough to talk about one bug. :)
>
> We could save some time if we have someone monitoring the bugs/feature and
> publish the high priority item into a report - something similar to what
> Keystone does [1].  Reviewers can look this up every time if they need to
> prioritize their reviews.
>
> We can rotate this responsibility among cores every month - even non-core
> if someone wants to volunteer.
>
> -Lin
>
> [1]
> https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting#Keystone_Weekly_Bug_Reports
>
>
>
>
> On Mon, Sep 28, 2015 at 7:22 PM, Tripp, Travis S 
> wrote:
>
>> Things always move more quickly at the end of a cycle because people feel
>> release pressure, but I do think this is a good idea. 2 - 3 minutes isn’t
>> very realistic. It would need to be planned for longer.
>>
>>
>>
>>
>>
>> On 9/28/15, 3:57 AM, "Rob Cresswell (rcresswe)" 
>> wrote:
>>
>> >Hi folks,
>> >
>> >I¹m wondering if we could try marking out a small 2-3 minute slot at the
>> >start of each weekly meeting to highlight Critical/ High bugs that have
>> >code up for review, as well as important blueprints that have code up for
>> >review. These would be blueprints for features that were identified as
>> >high priority at the summit.
>> >
>> >The thought here is that we were very efficient in L-RC1 at moving code
>> >along, which is nice for productivity, but not really great for
>> stability;
>> >it would be good to do this kind of targeted work earlier in the cycle.
>> >I¹ve noticed other projects doing this in their meetings, and it seems
>> >quite effective.
>> >
>> >Rob
>> >
>> >
>>
>> >__
>> >OpenStack Development Mailing List (not for usage questions)
>> >Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] Liberty release - what is the correct version - 2015.2.0, 8.0.0 or 12.0.0?

[Richard Jones]

Hi all,

Historically OpenStack releases are versioned .N as documented in the
Release Naming wiki page[1]. The Liberty Release Schedule[2] has the
version 2015.2.0. However, in Horizon land, I've been informed that
OpenStack is moving to semver. I can't find any information about this move
except a blog post by Doug Hellmann[3]. Is that correct? What is the
version string supposed to be: 8.0.0 as has been used in Horizon's
documentation[4], or 12.0.0 as hinted at by the blog post? I believe that
if we are moving to semver, then 12.0.0 is appropriate.


  Richard

[1] https://wiki.openstack.org/wiki/Release_Naming
[2] https://wiki.openstack.org/wiki/Liberty_Release_Schedule
[3]
https://doughellmann.com/blog/2015/05/29/openstack-server-version-numbering/
[4]
https://raw.githubusercontent.com/openstack/horizon/master/doc/source/topics/settings.rst
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] Selenium is now green - please pay attention to it

[Richard Jones]

Hi folks,

Selenium tests "gate-horizon-selenium-headless" are now green in master
again.

Please pay attention if it goes red. I will probably notice, but if I
don't, and you can't figure out what's going on, please feel free to get in
touch with me (r1chardj0n3s on IRC in #openstack-horizon, or email). Let's
try to keep it green!


Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon]Informal Pre-summit get together

[Richard Jones]

I'm planning on wearing my openstack badge, to make things a little easier
for those I don't spend all week on hangouts with ;)

On Sun, 17 May 2015 at 16:33 Michael Krotscheck 
wrote:

> Do you have any distinguishing features? Like an enormous openstack hat?
>
> On Sun, May 17, 2015, 5:06 AM Doug Fish  wrote:
>
>> I missed the discussion in IRC. I'm going to be there Sunday. If we also
>> go out Monday I'll probably show up then too
>>
>> Doug
>>
>> > On May 17, 2015, at 12:40 AM, "Tripp, Travis S" 
>> wrote:
>> >
>> > Just to double check, there was some IRC discussion about Monday.  So is
>> > it still Sunday?
>> >
>> >> On 5/14/15, 3:41 PM, "Douglas Fish"  wrote:
>> >>
>> >>
>> >> The Horizon team will be meeting on Sunday night informally over a beer
>> >> and
>> >> dinner:
>> >> Sunday 6pm @  The Charles Bar http://thecharlesbar.ca/
>> >>
>> >> Hope to see you there!
>> >>
>> >> Doug
>> >>
>> >>
>> >>
>> __
>> >> OpenStack Development Mailing List (not for usage questions)
>> >> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>> >
>> >
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] dashboard-app split in horizon

[Richard Jones]

As part of the ongoing Horizon project code reorganisation, we today agreed
to clean up the Horizon-the-Framework and OpenStack Dashboard separation
issue by doing a couple of things:

1. nuke (the recently-created) horizon dashboard-app by moving the angular
app over to dashboard and the other contents to appropriate places (mostly
under the heading of "tech-debt" :)
2. move base.html, _conf.html and _scripts.html from horizon over to
dashboard.

Thanks to Cindy, Sean and Thai for the pair (er triple?) programming
keeping me honest today.

The first step is done and captured in several linked patches based off
your leaf patch "ngReorg - Create dashboard-app" <
https://review.openstack.org/#/c/184597/> (yes, I am nuking the thing
created by your patch).

I've not done the second step, but might find some time since I have 6
hours to waste in LAX tomorrow.


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] dashboard-app split in horizon

[Richard Jones]

As a follow-up to this [in the misguided hope that anyone will actually
read this conversation with myself ;-)] I've started looking at the
base.html split. At the summit last week, we agreed to:

1. move base.html over from the framework to the dashboard, and
2. move the _conf.html and _scripts.html over as well, since they configure
the application (dashboard).

Upon starting the work it occurs to me that all of the other files
referenced by base.html should also move. So, here's the complete list of
base.html components and whether they should move over in my opinion:

- horizon/_custom_meta.html
  Yep, is an empty file in horizon, intended as an extension point in
dashboard. The empty file (plus an added comment) should move.
  - horizon/_stylesheets.html
  Is just a dummy in horizon anyway, should move.
- horizon/_conf.html
  Yep, should move.
- horizon/client_side/_script_loader.html
  Looks to be a framework component not intended for override, so we should
leave it there.
- horizon/_custom_head_js.html
  Yep, is an empty file in horizon, intended as an extension point in
dashboard. Move, with a comment added.
- horizon/_header.html
  There is a basic implementation in framework but the real (used)
implementation is in dashboard, so should move.
- horizon/_messages.html
  This is a framework component, so I think should stay there. I'm not sure
whether anyone would ever wish to override this. Also the bulk of it is
probably going to be replaced by the  implementation anyway... hmm...
- horizon/common/_sidebar.html
  This is an overridable component that I think should move.
- horizon/common/_page_header.html
  This is an overridable component that I think should move.
- horizon/_scripts.html
  Yep, should move.

Thoughts, anyone who has read this far?


Richard


On Sat, 23 May 2015 at 11:46 Richard Jones  wrote:

> As part of the ongoing Horizon project code reorganisation, we today
> agreed to clean up the Horizon-the-Framework and OpenStack Dashboard
> separation issue by doing a couple of things:
>
> 1. nuke (the recently-created) horizon dashboard-app by moving the angular
> app over to dashboard and the other contents to appropriate places (mostly
> under the heading of "tech-debt" :)
> 2. move base.html, _conf.html and _scripts.html from horizon over to
> dashboard.
>
> Thanks to Cindy, Sean and Thai for the pair (er triple?) programming
> keeping me honest today.
>
> The first step is done and captured in several linked patches based off
> your leaf patch "ngReorg - Create dashboard-app" <
> https://review.openstack.org/#/c/184597/> (yes, I am nuking the thing
> created by your patch).
>
> I've not done the second step, but might find some time since I have 6
> hours to waste in LAX tomorrow.
>
>
>  Richard
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] configuration of plugins in an angularjs world

[Richard Jones]

Hi all,

Just a new conundrum for you :)

At the top level of our base.html dashboard app, we have some dependencies
'horizon.auth', 'horizon.framework' and 'hz.dashboard' (that name will
change to 'horizon.dashboard', but that's for another day). I think we need
to remove 'hz.dashboard' from that list of dependencies and the list needs
to be finer-grained *and configurable*.

Currently in our small dashboard angularjs module structure we have
hz.dashboard depend on all its sub-modules. This means that you can't
disable some dashboards. The JS files would no longer be loaded (because
they are in the "enabled" structure), but the dependency would remain
(because dependency lists are not).


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] dashboard-app split in horizon

[Richard Jones]

I have now submitted the patch to do the html shenanigans as a new step in
the current ngReorg pipeline (inserted because it should fix a test failure
popping up in the dashboard-app move final step).

https://review.openstack.org/#/c/186295/

Ryan's API reorg should probably depend on this patch also as that should
fix *its* test failures also.

And before anyone says anything, no I'm not particularly thrilled about the
new horizon/test/templates/base.html but frankly I'm not sure how else to
make it work. We could probably cull the JS from that file though. I'm
pretty sure none of the django unit tests exercise JS, and I believe
Selenium works off a different interface (but I've run out of time today to
investigate).


 Richard


On Thu, 28 May 2015 at 02:15 Thai Q Tran  wrote:

> Yes Rob, you are correct. ToastService was something Cindy wrote to
> replace horizon.alert (aka messages). We can't remove it because legacy
> still uses it.
>
> -"Rob Cresswell (rcresswe)"  wrote: -
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> From: "Rob Cresswell (rcresswe)" 
> Date: 05/26/2015 11:29PM
>
> Subject: Re: [openstack-dev] [Horizon] dashboard-app split in horizon
>
> Went through the files myself and I concur. Most of these files define
> pieces specific to our implementation of the dashboard, so should be moved.
>
>  I’m not entirely sure on where _messages should sit. As we move forward,
> won’t that file just end up as a  element and nothing more? Maybe
> I’m misinterpreting it, I’m not familiar with toastService.
>
>  Rob
>
>
>   From: Richard Jones 
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Date: Tuesday, 26 May 2015 01:35
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Cc: "Johanson, Tyr H" 
> Subject: Re: [openstack-dev] [Horizon] dashboard-app split in horizon
>
>   As a follow-up to this [in the misguided hope that anyone will actually
> read this conversation with myself ;-)] I've started looking at the
> base.html split. At the summit last week, we agreed to:
>
>  1. move base.html over from the framework to the dashboard, and
> 2. move the _conf.html and _scripts.html over as well, since they
> configure the application (dashboard).
>
>  Upon starting the work it occurs to me that all of the other files
> referenced by base.html should also move. So, here's the complete list of
> base.html components and whether they should move over in my opinion:
>
>  - horizon/_custom_meta.html
>   Yep, is an empty file in horizon, intended as an extension point in
> dashboard. The empty file (plus an added comment) should move.
>   - horizon/_stylesheets.html
>   Is just a dummy in horizon anyway, should move.
> - horizon/_conf.html
>   Yep, should move.
> - horizon/client_side/_script_loader.html
>   Looks to be a framework component not intended for override, so we
> should leave it there.
> - horizon/_custom_head_js.html
>Yep, is an empty file in horizon, intended as an extension point in
> dashboard. Move, with a comment added.
>  - horizon/_header.html
>   There is a basic implementation in framework but the real (used)
> implementation is in dashboard, so should move.
> - horizon/_messages.html
>   This is a framework component, so I think should stay there. I'm not
> sure whether anyone would ever wish to override this. Also the bulk of it
> is probably going to be replaced by the  implementation anyway...
> hmm...
> - horizon/common/_sidebar.html
>   This is an overridable component that I think should move.
> - horizon/common/_page_header.html
>   This is an overridable component that I think should move.
> - horizon/_scripts.html
>Yep, should move.
>
>  Thoughts, anyone who has read this far?
>
>
>  Richard
>
>
> On Sat, 23 May 2015 at 11:46 Richard Jones  wrote:
>
>>  As part of the ongoing Horizon project code reorganisation, we today
>> agreed to clean up the Horizon-the-Framework and OpenStack Dashboard
>> separation issue by doing a couple of things:
>>
>>  1. nuke (the recently-created) horizon dashboard-app by moving the
>> angular app over to dashboard and the other contents to appropriate places
>> (mostly under the heading of "tech-debt" :)
>> 2. move base.html, _conf.html and _scripts.html from horizon over to
>> dashboard.
>>
>>  Thanks to Cindy, Sean and Tha

Re: [openstack-dev] [horizon] Angular-Django i18n

[Richard Jones]

Hi Thai,

Thanks for writing that up! As I can tell the blueprint only has the
"whiteboard" which is kinda yuck, so here's my comments:

I'm already on the record for preferring markup approach 3 so we can avoid
having to write our own implementation of a translation engine. I really,
really don't want to see us write our own :/

I'd also like to see us use a minimum-effort approach for extraction and I
think approach 2 is that. It gives is a single point to invoke for message
catalog maintenance and doesn't require deployers to use a node.js tool for
message extraction. I believe Babel (http://babel.pocoo.org/) is the most
appropriate tool already existing for extracting the messages. There's
already a babel plugin for django, we would just need to write one for
angular-gettext. I believe this should allay concerns raised in extraction
approach 2.

And if you're going to do extraction approach 3 then you've practically
implemented a plugin for Babel so you might as well do approach 2 :)

I've looked into extraction approach 4 and got lost in a twisty maze of
ick. I'm not sure there's a clean way to extend Django's makemessage.


 Richard


On Fri, 29 May 2015 at 11:06 Thai Q Tran  wrote:

> Hi folks,
>
> Just drafted a blueprint on this topic so that we can all be on the same
> page. Its quite long, but hopefully you will take some time to read through
> it and provide some meaningful feedback. In the mean time, I will do some
> more investigation and keep you all posted. If you have questions or
> concerns, please contact me in IRC or post it in the comment section of the
> blueprint.
>
> Link to blueprint:
> https://blueprints.launchpad.net/horizon/+spec/angular-makemessages
>
> Thanks,
> Thai
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] high priority patches, please review

[Richard Jones]

Hi Horizon devs,

The following test patches are a high priority, blocking further new work
in Liberty until they're landed. Please consider helping review them to get
the landed ASAP:

https://review.openstack.org/#/c/170554/
https://review.openstack.org/#/c/167738/
https://review.openstack.org/#/c/172057/
https://review.openstack.org/#/c/178227/
https://review.openstack.org/#/c/176532/
https://review.openstack.org/#/c/178434/
https://review.openstack.org/#/c/167326/


Cheers,

 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [javascript] [horizon] [merlin] [refstack] Javascript Linting

[Richard Jones]

JSCS in Horizon has been extended with the John Papa style guidelines to
enforce consistent angularjs code style*. It's no longer just a findbug
tool. I don't have time to investigate - can ESLint perform the same role
for Horizon?

Current Horizon activity involves a whole lot of bringing code into line
with that style (and other JSCS check fails).


  Richard

* https://review.openstack.org/#/c/181311/

On Tue, 16 Jun 2015 at 09:40 Michael Krotscheck 
wrote:

> I'm restarting this thread with a different subject line to get a broader
> audience. Here's the original thread:
> http://lists.openstack.org/pipermail/openstack-dev/2015-June/066040.html
>
> The question at hand is "What will be OpenStack's javascript equivalent of
> flake8". I'm going to consider the need for common formatting rules to be
> self-evident. Here's the lay of the land so far:
>
>- Horizon currently uses JSCS.
>- Refstack uses Eslint.
>- Merlin doesn't use anything.
>- StoryBoard (deprecated) uses eslint.
>- Nobody agrees on rules.
>
> *JSCS*
> JSCS Stands for "JavaScript CodeStyle". Its mission is to enforce a style
> guide, yet it does not check for potential bugs, variable overrides, etc.
> For those tests, the team usually defers to (preferred) JSHint, or ESLint.
>
> *JSHint*
> Ever since JSCS was extracted from JSHint, it has actively removed rules
> that enforce code style, and focused on findbug style tests instead. JSHint
> still contains the "Do no evil" license, therefore is not an option for
> OpenStack, and has been disqualified.
>
> *ESLint*
> ESLint's original mission was to be an OSI compliant replacement for
> JSHint, before the JSCS split. It wants to be a one-tool solution.
>
> My personal opinion/recommendation: Based on the above, I recommend we use
> ESLint. My reasoning: It's one tool, it's extensible, it does both
> codestyle things and bug finding things, and it has a good license. JSHint
> is disqualified because of the license. JSCS is disqualified because it is
> too focused, and only partially useful on its own.
>
> I understand that this will mean some work by the Horizon team to bring
> their code in line with a new parser, however I personally consider this to
> be a good thing. If the code is good to begin with, it shouldn't be that
> difficult.
>
> This thread is not there to argue about which rules to enforce. Right now
> I just want to nail down a tool, so that we can (afterwards) have a
> discussion about which rules to activate.
>
> Michael
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [javascript] [horizon] [merlin] [refstack] Javascript Linting

[Richard Jones]

Sorry, wrong patch. That one added the style requirement to the project
contribution guidelines. This is the one that added the .jscsrc config:

https://review.openstack.org/#/c/185725/

On Tue, 16 Jun 2015 at 14:21 Richard Jones  wrote:

> JSCS in Horizon has been extended with the John Papa style guidelines to
> enforce consistent angularjs code style*. It's no longer just a findbug
> tool. I don't have time to investigate - can ESLint perform the same role
> for Horizon?
>
> Current Horizon activity involves a whole lot of bringing code into line
> with that style (and other JSCS check fails).
>
>
>   Richard
>
> * https://review.openstack.org/#/c/181311/
>
> On Tue, 16 Jun 2015 at 09:40 Michael Krotscheck 
> wrote:
>
>> I'm restarting this thread with a different subject line to get a broader
>> audience. Here's the original thread:
>> http://lists.openstack.org/pipermail/openstack-dev/2015-June/066040.html
>>
>> The question at hand is "What will be OpenStack's javascript equivalent
>> of flake8". I'm going to consider the need for common formatting rules to
>> be self-evident. Here's the lay of the land so far:
>>
>>- Horizon currently uses JSCS.
>>- Refstack uses Eslint.
>>- Merlin doesn't use anything.
>>- StoryBoard (deprecated) uses eslint.
>>- Nobody agrees on rules.
>>
>> *JSCS*
>> JSCS Stands for "JavaScript CodeStyle". Its mission is to enforce a style
>> guide, yet it does not check for potential bugs, variable overrides, etc.
>> For those tests, the team usually defers to (preferred) JSHint, or ESLint.
>>
>> *JSHint*
>> Ever since JSCS was extracted from JSHint, it has actively removed rules
>> that enforce code style, and focused on findbug style tests instead. JSHint
>> still contains the "Do no evil" license, therefore is not an option for
>> OpenStack, and has been disqualified.
>>
>> *ESLint*
>> ESLint's original mission was to be an OSI compliant replacement for
>> JSHint, before the JSCS split. It wants to be a one-tool solution.
>>
>> My personal opinion/recommendation: Based on the above, I recommend we
>> use ESLint. My reasoning: It's one tool, it's extensible, it does both
>> codestyle things and bug finding things, and it has a good license. JSHint
>> is disqualified because of the license. JSCS is disqualified because it is
>> too focused, and only partially useful on its own.
>>
>> I understand that this will mean some work by the Horizon team to bring
>> their code in line with a new parser, however I personally consider this to
>> be a good thing. If the code is good to begin with, it shouldn't be that
>> difficult.
>>
>> This thread is not there to argue about which rules to enforce. Right now
>> I just want to nail down a tool, so that we can (afterwards) have a
>> discussion about which rules to activate.
>>
>> Michael
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Angular dependencies and the hz.dashboard namespace

[Richard Jones]

The "Adding Angular Identity Dashboard"[1] patch exposed an issue I saw
previously that worried me[2].

I believe during recent Horizon work the concept of angular module
namespaces and dependencies have been conflated. There's this idea that if
you create a submodule inside a module namespace you *must* have that
module depend on that submodule. I believe that is incorrect - just look at
the angular codebase itself, and how it is used. If you want the ngCookies
module in a couple of places then you have those modules depend on
ngCookies (or, more likely, you just add it as a dependency to the app).
The point is it's not just added automatically to the "ng" module as a
dependency. If you need to use a module's functionality, you depend on the
module. You don't just have all your modules *automatically* pulled in as
dependencies. I have proposed a patch to remedy this for the existing
"optional"[3] project dashboard[4].

I believe it is unnecessary to add extension of the hz.dashboard dependency
list[5] since we already have an extensible dependency list for the
application itself (which the hz.dashboard dependency list just extends).

To answer the question explicitly raised "what is the point of having the
hz.dashboard module if it has no dependencies?" It does two things: it
defines the namespace in a formal manner (by itself unnecessary, but it's
still nice to do) and it defines a constant which is used by other code.
Eventually it may define more. There is an important difference between
Python modules and Angular modules here - using a Python module like
hz.dashboard in this way could cause problems because of the way Python
sub-module namespaces and import ordering work. Angular's modules work very
differently, and are not burdened by the same issues. In Python that
constant would most likely have to be pushed out to a sub-module to avoid
import loops.


 Richard

[1] https://review.openstack.org/190852
[2] https://bugs.launchpad.net/horizon/+bug/1466713
[3] There may be other issues that prevent the project dashboard being
optional - there are dependencies in Python-land from the admin dashboard
hard-coded over to the project dashboard, for example. It might be a good
idea to remedy that, since I think it probably exposes some other
structural issues in the plugin mechanism we're providing to deployers.
[4] https://review.openstack.org/#/c/193401/
[5] https://review.openstack.org/193671
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Angular dependencies and the hz.dashboard namespace

[Richard Jones]

On 20 June 2015 at 09:11, Thai Q Tran  wrote:

> -----Richard Jones  wrote: -
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> From: Richard Jones 
> Date: 06/19/2015 03:49PM
> Subject: [openstack-dev] Angular dependencies and the hz.dashboard
> namespace
>
>
> The "Adding Angular Identity Dashboard"[1] patch exposed an issue I saw
> previously that worried me[2].
>
> I believe during recent Horizon work the concept of angular module
> namespaces and dependencies have been conflated. There's this idea that if
> you create a submodule inside a module namespace you *must* have that
> module depend on that submodule. I believe that is incorrect - just look at
> the angular codebase itself, and how it is used. If you want the ngCookies
> module in a couple of places then you have those modules depend on
> ngCookies (or, more likely, you just add it as a dependency to the app).
> The point is it's not just added automatically to the "ng" module as a
> dependency. If you need to use a module's functionality, you depend on the
> module. You don't just have all your modules *automatically* pulled in as
> dependencies. I have proposed a patch to remedy this for the existing
> "optional"[3] project dashboard[4].
>
> I believe it is unnecessary to add extension of the hz.dashboard
> dependency list[5] since we already have an extensible dependency list for
> the application itself (which the hz.dashboard dependency list just
> extends).
>
> To answer the question explicitly raised "what is the point of having the
> hz.dashboard module if it has no dependencies?" It does two things: it
> defines the namespace in a formal manner (by itself unnecessary, but it's
> still nice to do) and it defines a constant which is used by other code.
> Eventually it may define more. There is an important difference between
> Python modules and Angular modules here - using a Python module like
> hz.dashboard in this way could cause problems because of the way Python
> sub-module namespaces and import ordering work. Angular's modules work very
> differently, and are not burdened by the same issues. In Python that
> constant would most likely have to be pushed out to a sub-module to avoid
> import loops.
>
>
>  Richard
>
> [1] https://review.openstack.org/190852
> [2] https://bugs.launchpad.net/horizon/+bug/1466713
> [3] There may be other issues that prevent the project dashboard being
> optional - there are dependencies in Python-land from the admin dashboard
> hard-coded over to the project dashboard, for example. It might be a good
> idea to remedy that, since I think it probably exposes some other
> structural issues in the plugin mechanism we're providing to deployers.
> [4] https://review.openstack.org/#/c/193401/
> [5] https://review.openstack.org/193671
>


> 1.
> Need to retain the same file structure so that pluggins continue to work.
> Example: https://github.com/stackforge/monasca-ui/tree/master/monitoring
>
> Basically we have existing pluggins that use this file structure, we need
> to honor it.
> This is not directly related to what we are talking about, but it does
> mean that we need to move static files out of
> /openstack_dashboard/static/MYDASHBOARD/* and into
> /openstack_dashboard/dashboards/MYDASHBOARD/static/*
>

Yep!



> Auto-discovery of static resources will need to also honor the pluggin
> model above, hence the file structure above.
> You will still need to manually define the ADD_ANGULAR_MODULES in your
> enabled file, auto-discovery doesn't know what you want enabled.
> Sean's patch is going to do that, but having some issues with SCSS.
> https://review.openstack.org/#/c/191592/
>

As far as I can tell that patch doesn't alter the current ADD_ANGULAR_MODULES
behaviour?



> hz.dashboard module will be empty because the hz.dashboard.MYDASHBOARD
> module will live at the app level via
> ADD_ANGULAR_MODULES. I would argue that it makes no sense to have an empty
> module, my preference is to just delete it.
> Constants are globally available in the app, something I think actually
> should be avoided, not encouraged.
>

OK, seems reasonable.



> Having hz.dashboard.tech-debt and workflow in the enabled file is not
> correct.
> They are core components needed by all dashboards and should be loaded by
> default, not via the pluggin mechanism.
>
> https://review.openstack.org/#/c/193401/4/openstack_dashboard/enabled/_10_project.py
>
> Lets say I have my own dashboard call MYDASHBOARD, and I decided to
> disable all other dashboards except mine,
> all of a sudden, things wil

[openstack-dev] [Horizon] eslint without color?

[Richard Jones]

I'm just curious why our eslint configuration (in packages.json) specifies
--no-color. It's much harder to spot the errors without color, and I always
end up running it manually to get the color. Also, karma output has color,
so why one and not the other?

In short, would anyone object to turning color on for eslint?


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] Testing of angular promise based code paths

[Richard Jones]

Hi all,

I've just added a comment to a review and I think I'd like to ask for a
broader discussion of whether I'm correct.

The review is here: https://review.openstack.org/#/c/284857/2

It boils down to: when testing code that uses a promise, should we *use* a
promise to have the follow-on callback invoked, or should we mock/spy and
then manually perform the same action the promise would if resolved?

The two forms are, broadly, pretending that a promise fired:

  it('successful submit calls the successCallback', function() {
*var successFunc = {success: angular.noop};*
*spyOn(successFunc, 'success');*
spyOn(nova, 'createKeypair')*.and.returnValue(successFunc);*
spyOn(toastService, 'add').and.returnValue({ add: angular.noop });
ctrl.submit();
*var successCallback = successFunc.success.calls.argsFor(0)[0];*
*var data = {name: 'newKeypair'};*
*successCallback(data);*
expect(toastService.add).toHaveBeenCalledWith(
'success',
'Successfully imported key pair newKeypair.'
);
  });

or actually using a promise and making it fire:

it('should load container contents', function test() {
  *var deferred = $q.defer();*
  spyOn(swiftAPI, 'getObjects')*.and.returnValue(deferred.promise);*
  service.selectContainer('spam');
  expect(service.containerName).toEqual('spam');
  expect(swiftAPI.getObjects).toHaveBeenCalledWith('spam', {delimiter:
'/'});
  *deferred.resolve({data: {items: ['two', 'items']}});*
  *$rootScope.$apply();*
  expect(service.objects).toEqual(['two', 'items']);
  expect(service.pseudo_folder_hierarchy).toEqual([]);
});


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Blueprints for Outreachy internship

[Richard Jones]

Hi Sayali,

Sorry I missed your original message, and thanks Matthias for responding.
Indeed the new Swift UI is currently being merged. Some of those blueprints
you list are things I'd like to see once the basic reimplementation is
landed. Select-all is already in the new version though (though we'd like
to see hotkey support in there). There's also a bunch of good work done by
the OpenStack UX team on the interface - changes that we'll look at rolling
in during the N cycle. My wishlist (also my TODO list), which doesn't have
blueprints yet, includes:

- https://github.com/chieffancypants/angular-hotkeys for cmd/ctrl-a
select-all
- https://github.com/danialfarid/ng-file-upload
- Recursive delete of folders
- replacing SimpleModals with bootstrap modals
- Support for >10k objects, most likely requiring Searchlight
- Support for >1k containers (filtering)
- Switch to action registry
- Implement UX’s design ideas
- Copy / rename of objects


  Richard

On 2 March 2016 at 00:07, Matthias Runge  wrote:

> On 29/02/16 12:41, Sayali Lunkad wrote:
> > Hi Matthias,
> >
> > Thanks for the links to the blueprints. I in fact did checkout the list
> > of bps earlier but I found many of them have not been approved yet. I
> > can give you a list of bps I would like to mentor for, from which we
> > could perhaps select one or two for the internship program depending on
> > which of these can be approved.
>
>
> Hey Sayali,
>
> >
> > Here is the list:
> >
> https://blueprints.launchpad.net/horizon/+spec/enable-download-multiple-objects
> > 
>
>
> Swift interface is currently changing quite quickly, see
> https://review.openstack.org/#/c/258769/ and related.
>
>
>
> >
> https://blueprints.launchpad.net/horizon/+spec/horizon-multiple-objects-uploading
> > 
>
> Same here, it's Horizon's swift interface, which changes right now.
> >
> https://blueprints.launchpad.net/horizon/+spec/horizon-reports-in-different-formats
> > 
>
> This is something, which still requires discussion; I strongly believe,
> Horizon is not a reporting tool at all.
>
> > https://blueprints.launchpad.net/horizon/+spec/swift-direct-upload
> > 
> Swift, same as above
>
>
> > https://blueprints.launchpad.net/horizon/+spec/instance-backup
> > 
>
> I'd support the idea, the blueprint is a bit thin, I would expect this
> to be implemented quickly.
>
>
> >
> https://blueprints.launchpad.net/horizon/+spec/select-all-checkbox-object-containers
>
> Swift, see above
>
>
> > https://blueprints.launchpad.net/horizon/+spec/per-user-limit-summary
>
> This is something, one could implement, yes. Still the blueprint is a
> bit thin though.
>
> Best,
> Matthias
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

We've solved *most* of the issues around releasing new xstatic packages,
documented here[1] (and related documentation).

We have one final issue that's blocking us, which is that during the
xstatic release there will be a point at which Horizon may be broken from
an integrated point of view - some of the interfaces may not work and fail
tests. The process goes something like this:

​Note: this assumes that depends-on can reliably bring in patches from all
over the place into a gate environment, which is technically possible, but
not necessarily correct today.

The problem is that because we can't atomically update both
upper-constraints *and* Horizon at the same time (or upper-constraints
*and* xstatic-angular, or Horizon *and* xstatic-angular) we run into a
situation where Horizon will be running against the wrong version of
xstatic-angular.

So we break one of the basic assumptions of the OpenStack world: that every
single commit in every repository for the integrated environment will pass
tests.

In the Python world, we code around this by making Horizon compatible with
both the X and X1 versions of xstatic-angular (if it was a Python library).
In Javascript land this is much more difficult - Javascript libraries tend
to break compatibility in far more interesting ways. Maintaining
compatibility across CSS and font releases is also a difficult problem,
though changes here are unlikely to break Horizon enough that gate tests
would fail. So, solution 1 to the problem is:

SOLUTION 1: always maintain Horizon compatibility across xstatic library
releases.

This is potentially very difficult to guarantee. So, a second solution has
been proposed:

SOLUTION 2: move the upper-constraints information for *the xstatic
libraries only* from the global upper-constraints file into Horizon's
repository.

This allows us to atomically update both Horizon and the xstatic library
version, removing the potential to break because of the version mismatch.
Unfortunately it means that we have version compatibility issues with
anything else that wants to install alongside Horizon that also uses
xstatic packages. For example, Horizon plugins. We could move Horizon
plugins into the Horizon repository to solve this. /ducks

A variation on this solution is:

SOLUTION 3: allow Horizon to locally override upper-constraints for the
time needed to merge a patch in devstack.

This solution allows Horizon to atomically update itself and the xstatic
library, but it also means installing Horizon in a CI/CD environment
becomes more difficult due to the need to always pull down the
upper-constraints file and edit it. We could code this in to tox, but that
doesn't help eg. devstack which needs to also do this thing.

SOLUTION 4: vendor the javascript

Heh.

SOLUTION 5: have dependencies on xstatic move from global requirements to
Horizon

This is similar to 2 & 3 with some of the same drawbacks (multiple users of
xstatic) but also we'd need a change to global-requirements handling to
ignore some entries in Horizon's requirements.

Your thoughts on any and all of the above are requested.


Richard

[1] https://review.openstack.org/#/c/289142/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

Two things I forgot to mention:

Currently there is another break point in the diagram of releases when X1
is released. Currently Horizon does not use upper-constraints and thus will
immediately pick up the new xstatic release file, potentially breaking
things. This is easy to fix - I will be proposing a patch soon.

SOLUTION 6 - make zuul capable of performing atomic cross-repository
commits.

Richard

Sent from my portable device, please excuse the brevity.
On 8 Mar 2016 15:52, "Richard Jones"  wrote:

> We've solved *most* of the issues around releasing new xstatic packages,
> documented here[1] (and related documentation).
>
> We have one final issue that's blocking us, which is that during the
> xstatic release there will be a point at which Horizon may be broken from
> an integrated point of view - some of the interfaces may not work and fail
> tests. The process goes something like this:
>
> ​Note: this assumes that depends-on can reliably bring in patches from all
> over the place into a gate environment, which is technically possible, but
> not necessarily correct today.
>
> The problem is that because we can't atomically update both
> upper-constraints *and* Horizon at the same time (or upper-constraints
> *and* xstatic-angular, or Horizon *and* xstatic-angular) we run into a
> situation where Horizon will be running against the wrong version of
> xstatic-angular.
>
> So we break one of the basic assumptions of the OpenStack world: that
> every single commit in every repository for the integrated environment will
> pass tests.
>
> In the Python world, we code around this by making Horizon compatible with
> both the X and X1 versions of xstatic-angular (if it was a Python library).
> In Javascript land this is much more difficult - Javascript libraries tend
> to break compatibility in far more interesting ways. Maintaining
> compatibility across CSS and font releases is also a difficult problem,
> though changes here are unlikely to break Horizon enough that gate tests
> would fail. So, solution 1 to the problem is:
>
> SOLUTION 1: always maintain Horizon compatibility across xstatic library
> releases.
>
> This is potentially very difficult to guarantee. So, a second solution has
> been proposed:
>
> SOLUTION 2: move the upper-constraints information for *the xstatic
> libraries only* from the global upper-constraints file into Horizon's
> repository.
>
> This allows us to atomically update both Horizon and the xstatic library
> version, removing the potential to break because of the version mismatch.
> Unfortunately it means that we have version compatibility issues with
> anything else that wants to install alongside Horizon that also uses
> xstatic packages. For example, Horizon plugins. We could move Horizon
> plugins into the Horizon repository to solve this. /ducks
>
> A variation on this solution is:
>
> SOLUTION 3: allow Horizon to locally override upper-constraints for the
> time needed to merge a patch in devstack.
>
> This solution allows Horizon to atomically update itself and the xstatic
> library, but it also means installing Horizon in a CI/CD environment
> becomes more difficult due to the need to always pull down the
> upper-constraints file and edit it. We could code this in to tox, but that
> doesn't help eg. devstack which needs to also do this thing.
>
> SOLUTION 4: vendor the javascript
>
> Heh.
>
> SOLUTION 5: have dependencies on xstatic move from global requirements to
> Horizon
>
> This is similar to 2 & 3 with some of the same drawbacks (multiple users
> of xstatic) but also we'd need a change to global-requirements handling to
> ignore some entries in Horizon's requirements.
>
> Your thoughts on any and all of the above are requested.
>
>
> Richard
>
> [1] https://review.openstack.org/#/c/289142/
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Proposal to add Diana Whitten to horizon-core

[Richard Jones]

+1 we need Diana's perspective, knowledge and enthusiasm

On 9 March 2016 at 10:03, Tripp, Travis S  wrote:

> +1, no questions asked. It is rare to find somebody with the passion that
> Diana has shown.
>
> From: Lin Hua Cheng mailto:os.lch...@gmail.com>>
> Reply-To: OpenStack List  openstack-dev@lists.openstack.org>>
> Date: Tuesday, March 8, 2016 at 3:48 PM
> To: OpenStack List  openstack-dev@lists.openstack.org>>
> Subject: Re: [openstack-dev] [horizon] Proposal to add Diana Whitten to
> horizon-core
>
> big +1 from me.
>
> She made a lot of contribution in making theming better for horizon and
> also prevented potential issues through reviews.
> Thanks for all the hard work, Diana.
>
> -Lin
>
> On Tue, Mar 8, 2016 at 2:06 PM, David Lyle  dkly...@gmail.com>> wrote:
> I propose adding Diana Whitten[1] to horizon-core.
>
> Diana is an active reviewer, contributor and community member. Over
> the past couple of releases, Diana has driven extensive changes around
> theme-ability in Horizon and drastically increased the standardization
> of our use of bootstrap. During this time, Diana has also provided
> valuable reviews to keep us on the straight and narrow preventing our
> continued abuse of defenseless web technologies.
>
> Please respond with comments, +1s, or objections by Friday.
>
> Thanks,
> David
>
> [1]
> http://stackalytics.com/?module=horizon-group&user_id=hurgleburgler&release=all
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

But Jeremy: Atomic Zuul. How cool is that name.


So I know I'm going to potentially get yelled at for voicing this, but what
are the chances that app-catalog and Horizon are ever installed on the same
system? That is, could it have its own xstatic constraints, with the
promise that the two constraints will never meet in a dark alley behind
some VM somewhere? app-catalog isn't in the integrated tests, so it's never
going to break the gate like Horizon will...


On 9 March 2016 at 11:07, Jeremy Stanley  wrote:

> On 2016-03-08 17:25:41 +1100 (+1100), Richard Jones wrote:
> [...]
> > SOLUTION 6 - make zuul capable of performing atomic cross-repository
> > commits.
>
> This seems unlikely to happen, as it's very much counter to Zuul's
> designed-in reliance on serializing changes to test before merging
> them.
> --
> Jeremy Stanley
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] Javascript linting and documentation

[Richard Jones]

Hey all,

I started looking into fixing the wall of "npm run lint" warnings today and
quickly noticed that about 85% of the "linting" warnings were about jsdoc.
We have significant issues around jsdoc anyway and we're supposed to be
using Sphinx anyway[1].

Some Horizon folks will know that I've been investigating generating
publishable documentation for our Javascript code for some time. Most of
the tools either don't work (dgeni) or produce pretty unusable output for a
project our size (jsdoc and grunt-ngdocs). I am about to investigate Sphinx
in collaboration with the docs team.

Regardless, I believe that the documentation generation should generate
errors about that documentation, not the code linter. Once we actually get
a documentation generator going. Until then, we don't even know what syntax
the documentation should follow.

I've proposed a change which just turns jsdoc "linting" off[2]. At the
moment, it is less than useful (the noise drowns out any other, legitimate
linting).


 Richard


[1] http://governance.openstack.org/reference/cti/javascript-cti.html
[2] https://review.openstack.org/#/c/290235/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

Hi Serg,

There's a crucial difference: getting the solution to this wrong for
Horizon will break the gate for all OpenStack projects. Updating an xstatic
package that a plugin uses has no side-effect outside of the plugin.


  Richard

On 10 March 2016 at 02:43, Serg Melikyan  wrote:

> >This is exactly my first thought. The plugins *must* depend on Horizon,
> and they have to use the same versions of XStatic packages anyways, so why
> specify the dependencies twice?
>
> Plugins may require different xstatic library, which is not even used
> by Horizon. Issue raised by Richard exists for plugins too, not only
> for Horizon itself.
>
>
> On Wed, Mar 9, 2016 at 12:24 AM, Radomir Dopieralski
>  wrote:
> > On 03/08/2016 11:43 PM, David Lyle wrote:
> >
> >> I'm wondering if since horizon is really the only project consuming
> >> these xstatic libraries and none are likely to venture down this path of
> >> madness with us that it would be safe to manage the xstatic requirements
> >> and upper-constraints locally.
> >>
> >> Technically the plugins for horizon depend on this, but they depend via
> >> horizon. If they require specific versions that are not supported by
> >> Horizon, I think all bets are off anyway.
> >
> >
> > This is exactly my first thought. The plugins *must* depend on Horizon,
> > and they have to use the same versions of XStatic packages anyways, so
> > why specify the dependencies twice? If the changes between versions are
> > so big as to be breaking, then the plugins have to be updated to work
> > with the new Horizon anyways.
> >
> > --
> > Radomir Dopieralski
> >
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
> http://mirantis.com | smelik...@mirantis.com
>
> +1 (650) 440-8979
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Javascript linting and documentation

[Richard Jones]

On 10 March 2016 at 06:49, Michael Krotscheck  wrote:

> I guess I don't see what problems is being solved by turning the rule off,
> and I also don't see the harm in having more check. It does generate a lot
> of warnings, but invoking `npm run lint -- --quiet` gets rid of those.
>

We already have a "lintq" npm task that does this, which most of us use.
The problem is that we then ignore all the legitimate code linting warnings.


Also, from my experience, sphinx-based doc builds are notoriously
> permissive about bad formatting. Eslint's stricter jsdoc checks would add
> an additional safety net.
>

My perspective on this is if the documentation builder can produce
documentation that is useful then it's enforcing exactly enough checks on
the input. For example, the jsdoc linter currently forces us to write
@returns statements in our docs for angular methods that have no return
value, and never will (i.e. module.config()) or is otherwise not exposed to
developers and not interesting (i.e. the return from service or controller
construction). I mentioned this in passing in the meeting, but most of our
JS documentation has been written to ngdoc syntax, and that's potentially a
good thing since it provides much greater hinting to the doc generators
about how to present and organise the output, but also influences things
like @returns usefulness. We just have to find the right tool (or, more
likely, create, since I've been looking for a while and haven't found a
suitable tool) that uses the information we're coding into our comments.



> However, if you're working on this with the docs team, then the result
> should be applicable to the entirety of openstack - meaning that once your
> work is complete, it may make sense to turn the rule off globally.
>

Yep!


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

On 10 March 2016 at 18:23, Matthias Runge  wrote:
>
> 4.alt.2:
> move to proper packages for static file management. I mean, they need to
> be built anyways.
>

Please define what you mean by "proper packages" here. I *think* you might
mean system packages (aka Debian or Red Hat) which is not feasible given
other environments that Horizon runs under. Please correct me if I'm wrong!



> It has been mentioned, xstatic packages can block the gate. We currently
> control xstatic package releases, thus we can roll back, if something
> goes wrong.
>
> If we're pulling directly with npm/bower, someone from the outside can
> break us. We already have the situation with pypi packages.
> With proper packages, we could even use the gate to release those
> packages and thus verify, we're not breaking anyone.
>

We're going to have potential breakage (gate breakage, in the integrated
tests) any time we release a package (regardless of release mechanism) and
have to update two separate repositories resulting in out-of-sync version
specification and expectation (ie. upper-constraints specification and
Horizon's code expectation) as described in my OP. The only solution that
we're aware of is to synchronise updating those two things, through one of
the mechanisms proposed so far (or possibly through a mechanism not yet
proposed.)


I think that David's proposal is the only really feasible approach at this
time:

1. Horizon maintains its own constrained version list for the xstatic
packages,
2. Plugins to Horizon must depend on Horizon to supply xstatic packages
except where they use additional packages that Horizon does not use, and
3. We avoid installing app-catalog (the system, not the plugin) in the
integrated tests (I don't believe doing this is even on the ... "horizon"
so to speak) *or* in a Debian / Red Hat (i.e. system-packaged) system that
also has Horizon installed. Or we try to convince app-catalog to stay
lock-step with Horizon's xstatic versions. I understand the risk of a
collision between app-catalog and Horizon in the same system-packaged
environment is very low.


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

On 10 March 2016 at 21:48, Beth Elwell  wrote:
>
> On 10 Mar 2016, at 07:46, Richard Jones  wrote:
>
>> It has been mentioned, xstatic packages can block the gate. We currently
>> control xstatic package releases, thus we can roll back, if something
>> goes wrong.
>>
>> If we're pulling directly with npm/bower, someone from the outside can
>> break us. We already have the situation with pypi packages.
>> With proper packages, we could even use the gate to release those
>> packages and thus verify, we're not breaking anyone.
>>
>
> We're going to have potential breakage (gate breakage, in the integrated
> tests) any time we release a package (regardless of release mechanism) and
> have to update two separate repositories resulting in out-of-sync version
> specification and expectation (ie. upper-constraints specification and
> Horizon's code expectation) as described in my OP. The only solution that
> we're aware of is to synchronise updating those two things, through one of
> the mechanisms proposed so far (or possibly through a mechanism not yet
> proposed.)
>
> If we will anyway have potential breakage I don’t understand why the
> better solution here would not be to just use the bower and npm tools which
> are standardised for JavaScript and would move Horizon more towards using
> widely recognised tooling from within not just Openstack but the wider
> development community. Back versions always need to be supported for a
> time, however I would add that long term this could end up saving time and
> create a stable longer term solution.
>

I (and others) have argued several times for this, for a number of reasons,
and it remains my preferred solution, but it has been shot down many times
:-(

There are three major hurdles that I recall (sorry if I forgot any, it's
getting late here):

1. system packaging; installers using Debian or Red Hat (completely
offline) installation will not be able to install Horizon. We don't have a
solution for this though various caching mechanisms have been proposed.
2. security; the bower installation mechanism is seen as being very
insecure - not that I would call the current xstatic mechanism secure. It's
all down to degrees, I suppose.
3. installation in the gate; I believe that currently installation from
bower would not be possible in the gate. This is pretty closely related to
#1.

I think I recall licensing came up at one point too, but I might be
mis-remembering.


  Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] PTL noncandidacy

[Richard Jones]

Thanks for everything David. My involvement and engagement with the project
would not be what it is without your leadership. I look forward to
continuing to work with and learn from you!


   Richard

On 12 March 2016 at 04:19, David Lyle  wrote:

> After five cycles as PTL of Horizon, I've decided not to run for the
> Newton cycle.
>
> I am exceptionally proud of the things we've accomplished over this
> time. I'm amazed by how much our project's community has grown and
> evolved.
>
> Looking at the community now, I believe we have a tremendous group of
> contributors for moving forward. There are several people capable of
> becoming great PTLs and overall the community will be healthier with
> some turnover in the PTL role. I feel honored to have had your trust
> over this time and lucky to have worked with such great people.
>
> I will still be around and will help the next PTL make a smooth
> transition where requested.
>
> David
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev