Re: [Crowbar] how to debug Travis failures

[James Tan]

On 03/12/2013 11:16 PM, andi_a...@dell.com wrote:

-Original Message-
From: crowbar-bounces On Behalf Of Adam Spiers
Sent: Monday, March 11, 2013 7:28 PM
To: crowbar
Subject: Re: [Crowbar] how to debug Travis failures

Adam Spiers (aspi...@suse.com) wrote:

I'm really keen that I (or James, or anyone else) don't become the
bottleneck on Travis builds.  I really think it needs to be a
team-wide responsibility, e.g. if anyone sees test failures and thinks
they may have caused them, that they take the lead in resolving them.

Yup.. as you might notice from the latest set of activity, there's been a 
concerted effort to fix the test failures.


Yes we noticed that, awesome!


One thing that caused some confusion is the distinction between:
a) the machinery that runs the tests - i.e travis itself,
b) the update-git cron job, and its activities
c) the actual test failures.

AFAIK:
we're not going to mess with travis proper :), so that takes care of a).
update-git ( b) above. If I understand correctly, you have it covered - in some 
remote corner of your lab there's a server that dutifully runs this on a 5 
minute interval? At this point, we're not running a similar process, making 
your server the only trigger for updating the travis repo. (so if it was 
running on your laptop and you went home for the weekend, no tests will be 
triggered till you were back ;).

Should we setup a mirror of this process, to avoid a SPOF? It appears 
relatively safe to run multiple copies - or do you suspect that Bad Things will 
happen (e.g. the 2 instances resulting in some sort badness ?). It appears that 
the script does a 'git push -f' so merge conflicts are not likely.


The update-git cron job was running in a VM on my workstation (as a 
stop-gap measure). However it stopped working properly a couple of weeks 
back while I was away, and so Adam has moved it to
http://ci.opensuse.org/job/crowbar-travis_ci-trackupstream/. We'll be 
updating the documentation in github.com soon to reflect this.



As for c)... yes. Definitely. I think we've been doing that.


+1000

Cheers,
James T.
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] [Fixed] crowbar/travis-ci-crowbar#53 (master - f8bdf12)

[James Tan]

On 03/13/2013 01:12 AM, Travis-CI wrote:



  The build was fixed.

Repository  crowbar/travis-ci-crowbar
Build #53   https://travis-ci.org/crowbar/travis-ci-crowbar/builds/5456934
Changeset 
https://github.com/crowbar/travis-ci-crowbar/compare/3c18df5c1c60...f8bdf1299fba 


Commit  f8bdf12 (master)
Message Merge pull request #7 from aabes/master

attemt to fix the missing rubygem/format
Author  aabes
Duration8 minutes and 10 seconds



Andi - Thanks for fixing Travis!

Cheers,
James T.
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

[Crowbar] removing non-engine files?

[Adam Spiers]

cdearborn (notificati...@github.com) wrote:
> Yes (I can't wait!).  This helper should only be in the network
> barclamp.  Rob checked a copy of it into the crowbar barclamp as a
> stop-gap to make the unit tests run.  Wayne is working on updating
> dev setup-unit-tests or barclamp_install or whatever to correctly
> copy any test helpers that are defined in the engines to the correct
> dev test location.  Once he finishes that work, we can delete this
> helper from the crowbar barclamp.

Great.  Talking of duplicated code, it seems that

  crowbar_engine/barclamp_$BC/

is a modified and restructured copy of

  crowbar_framework/

for some or all of the barclamp engines.  Whilst I know we still need
to keep one or two of the old files in the very short term, such as
the network helper referenced above, can't we already remove most of
the non-engine files?  Having a whole load of duplicated code hanging
around risks someone reading or changing the wrong version by mistake,
or worse, the old code actually being used somewhere ... sorry for
being annoying or if I misunderstood something, but I'm even more
religious about DRY than I am about emacs ;-)

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] testing and pull requests

[Adam Spiers]

Andi Abes (andi_a...@dell.com) wrote:
> > -Original Message-
> > From: crowbar-bounces On Behalf Of Adam Spiers
> > Sent: Monday, March 11, 2013 2:24 PM
> > To: crowbar
> > Subject: [Crowbar] testing and pull requests
> > 
> > "What testing do we run before submitting?" is a key question; thanks for
> > bringing it up.  Unfortunately the answer is currently "only whatever we run
> > manually", since Travis only runs on a merged repository containing master
> > from each barclamp repository, due to barclamps currently being unable to 
> > run
> > their unit tests standalone.
> > (There are Trello cards to address this, but it's unlikely to get fixed 
> > soon.)
> 
> The general pattern we've been following is to have barclamps extend 
> framework provided classes, and that code currently resides in 
> crowbar/crowbar_framework... so to get to a point where we can run unit tests 
> for standalone barclamps will require copying in the framework (chicken, 
> meet egg...).

Hmm ... hopefully there is an industry-wide best practice for reusing
shared code between engines which we can follow?

> > That means we all need to be as conscientous as possible by running tests
> > before every pull request, otherwise master is likely to break often.  I 
> > just spoke
> > with James and agree with his proposal to prototype barclamps as gems which
> > can run their unit tests standalone (I get the impression engines should 
> > help
> > with this but I am not an expert in the area).  That would allow Travis to 
> > run on
> > pull requests to individual barclamp repos.
> 
> A possible short term approach that would give us testing prior to
> pull requests being merged would be to reflect Pull requests from
> the different repo, into the travis repo.

Right, I created https://trello.com/c/6qn6cd0z for this a few weeks ago.
Sorry if I didn't publicise this enough at the time.

> How is this different from what's happening now?
> Now, the update-git script checks for changed files between the travis repo 
> (as checked-out locally on the server running the script) and the main 
> crowbar repo. If it finds differences, it pushes an update to the 
> travis-ci-crowbar repo.
> This process will only catch all the merged pull requests.
> 
> The approach I proposing works differently:
> On a periodic basis, pull the crowbar repos (crowbar and each barclamp) for 
> pending pull requests. For each open pull request, create a pull request 
> against the travis-ci-crowbar repo. This in turn will trigger travis to test 
> the pull request, before it is actually merged.

No, I think we want to stick with handling bundles in a single pass,
so that each Travis pull request corresponds to a bundle, or to a
singleton if there is no bundle.

> There are a few complications that need to be handled (but I believe a 
> solution exists for both):
> a) Bundles - often there are sets of pull requests against different 
> barclamps (or framework + barclamp) that will only pass tests if 
>  they're evaluated as a 'bundle' (i.e. both sets of changes are required). 

I already outlined a solution for this in the Trello card and in fact
even took the first steps towards implementing.  One approach is to
iterate over the ids provided by "./dev ci open-ids" and ensure that
for each, a branch called (say) pull-req/$ID exists.  If not, create
it via "./dev pull-requests switch" followed by the normal rsync.
Then use something like "hub" to issue the pull request to the Travis
repo.  I have already forked the repo to the Travis commit-bot's
account to allow this.

The other approach is to merge into a non-flat tree using git subtree,
and perform the tree merging at test-time by telling Travis to do the
"./dev tests setup" bit.  This has the advantage of preserving commit
history, authorship etc.

See also https://trello.com/c/tqCDVLgH

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] [Fixed] crowbar/travis-ci-crowbar#53 (master - f8bdf12)

[Andi_Abes]

I just did the little last one push...
Chris Wayne Dave and others did the fixing

From: James Tan [mailto:ja...@suse.de]
Sent: Wednesday, March 13, 2013 4:12 AM
To: Abes, Andi
Cc: crowbar
Subject: Re: [Crowbar] [Fixed] crowbar/travis-ci-crowbar#53 (master - f8bdf12)

On 03/13/2013 01:12 AM, Travis-CI wrote:
The build was fixed.


Repository

crowbar/travis-ci-crowbar

Build #53

https://travis-ci.org/crowbar/travis-ci-crowbar/builds/5456934

Changeset

https://github.com/crowbar/travis-ci-crowbar/compare/3c18df5c1c60...f8bdf1299fba



Commit

f8bdf12 (master)

Message

Merge pull request #7 from aabes/master

attemt to fix the missing rubygem/format

Author

aabes

Duration

8 minutes and 10 seconds


Andi - Thanks for fixing Travis!

Cheers,
James T.
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

[Crowbar] Rubygems 2.0 - breaks bundle and others

[Andi_Abes]

If you're about to install a new build env, or building with internet wide 
downloads, keep reading. Otherwise it's safe to ignore.

The latest version of RubyGems (2.0.3) has a bunch of breaking changes, some 
break bundler, others break chef - lots of fun to be had.

The last safe version on the 1.8 train is 1.8.25. To get it:
gem update --system 1.8.25

more details here: http://blog.rubygems.org/2013/02/24/2.0.0-released.html

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

[Crowbar] Loadbalancer barclamp

[Andi_Abes]

A little while back, I started playing with a loadbalancer barclamp, and found 
some interesting options out there that can provide some nice features, namely:

* HA

* Scalability

* SSL termination

To be clear, I'm not suggesting that this barclamp will land in the CB2.0 
timeframe, but some folks are eager to "get them one of themz LB's".

This is also separate from the buzz around LBaaS that is being built as a 
Quantum service. The latter is meant to be Cloud user facing, and load balance 
cloud workloads. This barclamp is meant to provide its cool features to the 
cloud infrastructure.

It's early days, but some folks will start beating on it (based on 
crowbar1.5)so  I thought it's time to bring into the sunlight (as a totally 
irrelevant note, I stole the term from the guys at [2]). Checkout the readme at 
[1] - there are links to the underlying tech I'm proposing to we use.



[1] https://github.com/crowbar/barclamp-loadbalancer

[2] http://sunlightfoundation.com/


___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] Loadbalancer barclamp

[Rob_Hirschfeld]

Thanks Andi - this is an important addition to helping get Grizzly 
infrastructure and Crowbar deployments to be more robust.

From: crowbar-bounces On Behalf Of Abes, Andi
Sent: Wednesday, March 13, 2013 9:19 AM
To: crowbar
Subject: [Crowbar] Loadbalancer barclamp

A little while back, I started playing with a loadbalancer barclamp, and found 
some interesting options out there that can provide some nice features, namely:

* HA

* Scalability

* SSL termination

To be clear, I'm not suggesting that this barclamp will land in the CB2.0 
timeframe, but some folks are eager to "get them one of themz LB's".

This is also separate from the buzz around LBaaS that is being built as a 
Quantum service. The latter is meant to be Cloud user facing, and load balance 
cloud workloads. This barclamp is meant to provide its cool features to the 
cloud infrastructure.

It's early days, but some folks will start beating on it (based on 
crowbar1.5)so  I thought it's time to bring into the sunlight (as a totally 
irrelevant note, I stole the term from the guys at [2]). Checkout the readme at 
[1] - there are links to the underlying tech I'm proposing to we use.



[1] https://github.com/crowbar/barclamp-loadbalancer

[2] http://sunlightfoundation.com/


___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] installing Fred onto PowerEdge M710HD

[Judd Maltin]

You might want to talk to your Dell representative about that.


On Tue, Mar 12, 2013 at 12:06 PM, Tim Cook  wrote:

> I am having issues with installing crowbar onto a PowerEdge M710HD blade .
>
> error:
>
> Some of your hardware needs non-free firmware files to operate. the
> firmware can be loaded from removable media, such as a USB stick or floppy.
>
> The missing firmware files are: q12500_fw.bin
>
> I have been unsuccessful at finding this driver anywhere including Dells
> support page for this product.
>
> Any ideas ?
>
> --
> Do not allow yourselves to be deceived: Great Minds are Skeptical.
> (Friedrich Nietzsche)
>
> ___
> Crowbar mailing list
> Crowbar@dell.com
> https://lists.us.dell.com/mailman/listinfo/crowbar
> For more information: http://crowbar.github.com/
>



-- 
Judd Maltin
T: 917-882-1270
F: 501-694-7809
what could possibly go wrong?
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] removing non-engine files?

[Wayne_Allen]

I would like to keep the old versions around for quick reference and delete 
them as each barclamp is fully integrated into the new system.

Thanks,
W

-Original Message-
From: crowbar-bounces On Behalf Of Adam Spiers
Sent: Wednesday, March 13, 2013 4:53 AM
To: crowbar
Subject: [Crowbar] removing non-engine files?

cdearborn (notificati...@github.com) wrote:
> Yes (I can't wait!).  This helper should only be in the network 
> barclamp.  Rob checked a copy of it into the crowbar barclamp as a 
> stop-gap to make the unit tests run.  Wayne is working on updating dev 
> setup-unit-tests or barclamp_install or whatever to correctly copy any 
> test helpers that are defined in the engines to the correct dev test 
> location.  Once he finishes that work, we can delete this helper from 
> the crowbar barclamp.

Great.  Talking of duplicated code, it seems that

  crowbar_engine/barclamp_$BC/

is a modified and restructured copy of

  crowbar_framework/

for some or all of the barclamp engines.  Whilst I know we still need to keep 
one or two of the old files in the very short term, such as the network helper 
referenced above, can't we already remove most of the non-engine files?  Having 
a whole load of duplicated code hanging around risks someone reading or 
changing the wrong version by mistake, or worse, the old code actually being 
used somewhere ... sorry for being annoying or if I misunderstood something, 
but I'm even more religious about DRY than I am about emacs ;-)

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] [barclamp-crowbar] add a rake task to run the travis tests in one go, so we can more easily... (#464)

[Adam Spiers]

aabes (notificati...@github.com) wrote:
> You can merge this Pull Request by running:
> 
>   git pull https://github.com/aabes/barclamp-crowbar rake-travis
> 
> Or you can view, comment on it, or merge it online at:
> 
>   https://github.com/crowbar/barclamp-crowbar/pull/464
> 
> -- Commit Summary --
> 
>   * add a rake task to run the travis tests in one go, so we can more easily 
> do that locally
> 
> -- File Changes --
> 
> M crowbar_framework/Rakefile (5)
> 
> -- Patch Links --
> 
> https://github.com/crowbar/barclamp-crowbar/pull/464.patch
> https://github.com/crowbar/barclamp-crowbar/pull/464.diff

Posthumous +1 for the great idea!  However there is a caveat that
`db:drop` with `RAILS_ENV=development` will drop not only
`development.sqlite3` but also `test.sqlite3`.  That means that
running "rake travis" with the development environment (the default)
will break your test environment.  I very recently tweaked ./dev's
reload_test_env() to cater for this but I'm not sure what the best
approach here is.  Having said that, the test environment is in a
dedicated tree in /tmp so maybe it doesn't matter too much?  Ideas:

- exclude `db:drop` from this new task
- override db:drop with something which only drops the current
  environment's db
- issue some kind of warning / ask for confirmation if interactive
- do nothing and try to be aware of this corner case

I believe there are only 3 use cases for this task:

1) Called by Travis via .travis.yml
2) Called by dev's reload_test_env()
3) Called manually from the CLI

This new task lets us eliminate small amount of duplication between
.travis.yml and ./dev, which is cool.

Any other use cases?  FWIW 1) is always non-interactive, 2) is
typically interactive but could be non-interactive, and 3) is almost
always interactive.

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

[Crowbar] Attrib State vs Deployment State - potentially code in transition

[Rob_Hirschfeld]

All,

We've been making a lot of progress on Deployments and deployment states 
(proposed, committed, active) as reflected by different snapshots.

In earlier designed, we'd captured something similar in the Attrib values 
(proposed vs actual).  Differences between these values are reflected as the 
Attrib state.

It looks like these two approaches are in conflict and we'll need to start 
resolving them over the next few days.  I do not expect this to be a sweeping 
change.

Basically, I'm proposing that general Attrib.state will be a reflection to the 
state of the snapshot that it is connected with.  For example, a snapshot that 
is the Deploy's proposed snapshot should have attribs that are also in state 
proposed.  This would be true for committed and active as well.

We have a few Attribs (HasNode specifically) that will still have their own 
state definition.

I also expect that we'll drop the "requested" component of the attribs since 
that information is also redundant with the active/commited/proposed state.  
This will be clearer as we incorporate inbound state from the jig.

Rob
__
Rob Hirschfeld
Distinguished Cloud Solution Architect
Dell | Cloud Edge, Data Center Solutions
blog robhirschfeld.com, twitter @zehicle
Please note, I am based in the CENTRAL (-6) time zone

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] [barclamp-crowbar] add a rake task to run the travis tests in one go, so we can more easily... (#464)

[Andi_Abes]

> -Original Message-
> From: crowbar-bounces On Behalf Of Adam Spiers
> Sent: Wednesday, March 13, 2013 12:17 PM
> To: crowbar
> Subject: Re: [Crowbar] [barclamp-crowbar] add a rake task to run the travis
> tests in one go, so we can more easily... (#464)
> 
> aabes (notificati...@github.com) wrote:
> > You can merge this Pull Request by running:
> >
> >   git pull https://github.com/aabes/barclamp-crowbar rake-travis
> >
> > Or you can view, comment on it, or merge it online at:
> >
> >   https://github.com/crowbar/barclamp-crowbar/pull/464
> >
> > -- Commit Summary --
> >
> >   * add a rake task to run the travis tests in one go, so we can more
> > easily do that locally
> >
> > -- File Changes --
> >
> > M crowbar_framework/Rakefile (5)
> >
> > -- Patch Links --
> >
> > https://github.com/crowbar/barclamp-crowbar/pull/464.patch
> > https://github.com/crowbar/barclamp-crowbar/pull/464.diff
> 
> Posthumous +1 for the great idea!  However there is a caveat that `db:drop`
> with `RAILS_ENV=development` will drop not only `development.sqlite3` but
> also `test.sqlite3`.  That means that running "rake travis" with the 
> development
> environment (the default) will break your test environment.  

Thanks for the heads up !

>I very recently
> tweaked ./dev's
> reload_test_env() to cater for this but I'm not sure what the best approach 
> here
> is.  Having said that, the test environment is in a dedicated tree in /tmp so
> maybe it doesn't matter too much?  Ideas:

I think it doesn't matter too too much... 

> 
> - exclude `db:drop` from this new task
> - override db:drop with something which only drops the current
>   environment's db
> - issue some kind of warning / ask for confirmation if interactive
> - do nothing and try to be aware of this corner case
> 
> I believe there are only 3 use cases for this task:
> 
> 1) Called by Travis via .travis.yml
> 2) Called by dev's reload_test_env()
> 3) Called manually from the CLI
> 
> This new task lets us eliminate small amount of duplication between 
> .travis.yml
> and ./dev, which is cool.
> 
> Any other use cases?  FWIW 1) is always non-interactive, 2) is typically
> interactive but could be non-interactive, and 3) is almost always interactive.
> 

My thinking was that this would be a good way to keep the 2 envs in sync 
(travis and dev tests) as we continue to tweak the testing to improve coverage. 
(e.g. futzzing with ways to get the BDD tests to run on travis).


> ___
> Crowbar mailing list
> Crowbar@dell.com
> https://lists.us.dell.com/mailman/listinfo/crowbar
> For more information: http://crowbar.github.com/

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] installing Fred onto PowerEdge M710HD

[Gangur, Hrushikesh (QA CloudOS)]

This resolved my issue:
http://archive.ubuntu.com/ubuntu/pool/main/l/linux-firmware/linux-firmware_1.79_all.deb

From: crowbar-boun...@dell.com [mailto:crowbar-boun...@dell.com] On Behalf Of 
Judd Maltin
Sent: Wednesday, March 13, 2013 7:27 AM
To: Tim Cook
Cc: Crowbar@dell.com
Subject: Re: [Crowbar] installing Fred onto PowerEdge M710HD

You might want to talk to your Dell representative about that.

On Tue, Mar 12, 2013 at 12:06 PM, Tim Cook 
mailto:tcoo...@gmail.com>> wrote:
I am having issues with installing crowbar onto a PowerEdge M710HD blade .

error:

Some of your hardware needs non-free firmware files to operate. the firmware 
can be loaded from removable media, such as a USB stick or floppy.

The missing firmware files are: q12500_fw.bin

I have been unsuccessful at finding this driver anywhere including Dells 
support page for this product.

Any ideas ?

--
Do not allow yourselves to be deceived: Great Minds are Skeptical.
(Friedrich Nietzsche)

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/



--
Judd Maltin
T: 917-882-1270
F: 501-694-7809
what could possibly go wrong?

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] [Still Failing] crowbar/travis-ci-crowbar#60 (master - ac51708)

[James Tan]

On 03/13/2013 06:38 PM, Travis-CI wrote:



  The build is still failing.

Repository  crowbar/travis-ci-crowbar
Build #60   https://travis-ci.org/crowbar/travis-ci-crowbar/builds/5477613



Anyone looking into this failure on Ruby 1.9.3? Details here:

https://travis-ci.org/crowbar/travis-ci-crowbar/jobs/5477618/#L562

Mostly an issue of incorrectly using "assert" when you really want to 
have "assert_equal". Please also run tests on Ruby 1.9.3, as Ruby 1.8.7 
is going to be dropped by upstream in June.


Thanks,
James T.
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] Porting core cookbooks to attribute injection

[James Tan]

On 03/07/2013 11:13 AM, Vincent Untz wrote:
> Hi,
>
> First, thanks for the overview on attribute injection in yesterday's
> call. This was quite useful to better understand what this all means and
> why we are moving this way, as well as how things can look like (with
> the keystone cookbook example).
>
> Ideally, we'd have a good example of porting a cookbook of a core
> barclamp to AI, which people can use as model when porting the other
> cookbooks. Is anybody working on such an example?

Agreed, a reference implementation would be very useful. Do we already 
have something like that? If not, let's pick one and make it so?

Cheers,
James T.


___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] [Still Failing] crowbar/travis-ci-crowbar#60 (master - ac51708)

[Rob_Hirschfeld]

Sigh  I'll correct.

Still working on the dev 1.9.3 migration over here.

From: crowbar-bounces On Behalf Of James Tan
Sent: Wednesday, March 13, 2013 12:54 PM
To: crowbar
Subject: Re: [Crowbar] [Still Failing] crowbar/travis-ci-crowbar#60 (master - 
ac51708)

On 03/13/2013 06:38 PM, Travis-CI wrote:
The build is still failing.


Repository

crowbar/travis-ci-crowbar

Build #60

https://travis-ci.org/crowbar/travis-ci-crowbar/builds/5477613


Anyone looking into this failure on Ruby 1.9.3? Details here:

https://travis-ci.org/crowbar/travis-ci-crowbar/jobs/5477618/#L562

Mostly an issue of incorrectly using "assert" when you really want to have 
"assert_equal". Please also run tests on Ruby 1.9.3, as Ruby 1.8.7 is going to 
be dropped by upstream in June.

Thanks,
James T.
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] [Still Failing] crowbar/travis-ci-crowbar#60 (master - ac51708)

[Christopher_Dearborn]

Guys,

I'm looking at this issue right now, and the problem is not an assert 
statement.  Those were fixed in an earlier pull request.  I know what the issue 
is and am working on an email now.


Thanks,

Chris

From: crowbar-bounces On Behalf Of Hirschfeld, Rob
Sent: Wednesday, March 13, 2013 2:01 PM
To: ja...@suse.de; crowbar
Subject: Re: [Crowbar] [Still Failing] crowbar/travis-ci-crowbar#60 (master - 
ac51708)

Sigh  I'll correct.

Still working on the dev 1.9.3 migration over here.

From: crowbar-bounces On Behalf Of James Tan
Sent: Wednesday, March 13, 2013 12:54 PM
To: crowbar
Subject: Re: [Crowbar] [Still Failing] crowbar/travis-ci-crowbar#60 (master - 
ac51708)

On 03/13/2013 06:38 PM, Travis-CI wrote:
The build is still failing.


Repository

crowbar/travis-ci-crowbar

Build #60

https://travis-ci.org/crowbar/travis-ci-crowbar/builds/5477613


Anyone looking into this failure on Ruby 1.9.3? Details here:

https://travis-ci.org/crowbar/travis-ci-crowbar/jobs/5477618/#L562

Mostly an issue of incorrectly using "assert" when you really want to have 
"assert_equal". Please also run tests on Ruby 1.9.3, as Ruby 1.8.7 is going to 
be dropped by upstream in June.

Thanks,
James T.
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] removing non-engine files?

[James Tan]

On 03/13/2013 04:54 PM, wayne_al...@dell.com wrote:
> I would like to keep the old versions around for quick reference and delete 
> them as each barclamp is fully integrated into the new system.

While I understand the convenience of keeping these files around during 
integration, it's really confusing for everyone else. Can you remove 
them and refer via the git history instead?

Thanks,
James T.

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] [Still Failing] crowbar/travis-ci-crowbar#60 (master - ac51708)

[James Tan]

On 03/13/2013 07:03 PM, christopher_dearb...@dell.com wrote:


I'm looking at this issue right now, and the problem is not an assert 
statement.  Those were fixed in an earlier pull request.  I know what 
the issue is and am working on an email now.




Okay, great to see that you guys are on it. I'll go stab on something 
else then.




*From:*crowbar-bounces *On Behalf Of *Hirschfeld, Rob
*Sent:* Wednesday, March 13, 2013 2:01 PM
*To:* ja...@suse.de; crowbar
*Subject:* Re: [Crowbar] [Still Failing] crowbar/travis-ci-crowbar#60 
(master - ac51708)


Sigh  I'll correct.

Still working on the dev 1.9.3 migration over here.



Anything I can help with?

Cheers,
James T.
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] Rubygems 2.0 - breaks bundle and others

[James Tan]

Hi Andi,

On 03/13/2013 01:14 PM, andi_a...@dell.com wrote:


If you're about to install a new build env, or building with internet 
wide downloads, keep reading. Otherwise it's safe to ignore.


The latest version of RubyGems (2.0.3) has a bunch of breaking 
changes, some break bundler, others break chef -- lots of fun to be had.


The last safe version on the 1.8 train is 1.8.25. To get it:

gem update --system 1.8.25

more details here: http://blog.rubygems.org/2013/02/24/2.0.0-released.html



Thanks for the heads up. This saved me a lot of time!

Cheers,
James T.
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] [Still Failing] crowbar/travis-ci-crowbar#60 (master - ac51708)

[Christopher_Dearborn]

I took a look at this issue, and the problem is that translation is not working 
properly in the Travis test setup.

A new migration was added that creates the default network barclamp deployment. 
 In that migration, I initially used Barclamp::DEFAULT_DEPLOYMENT_NAME for the 
name of the deployment.  I later used the same constant to look up the default 
deployment.  This failed in the dev test environment because the I18n.t call 
was failing during the database migration and defaulting to "default", while it 
was succeeding in the I18n.t call while the unit tests were running, which 
caused it to try to look up a deployment with the name "Default".  This caused 
the lookup to fail, and resulted in errors in the dev test environment that 
were identical to those seen in Travis.  To work around this problem (I think 
translation failing during DB migration is probably understandable - correct me 
if you think otherwise), I hard coded the Deployment name to "Default" in the 
DB migration, and everything worked.

What we are seeing in the Travis errors are exactly the opposite problem.  The 
network barclamp default deployment name is hard coded to "Default" in the DB 
migration.  When the Barclamp::DEFAULT_DEPLOYMENT_NAME constant is used while 
the unit tests are running, the I18n.t translation is failing and defaulting 
the name to "default".  As a result, the deployment lookup by name fails which 
causes the unit tests to fail.

At any rate, I can easily fix this by not using I18n.t() to get/set the name of 
the deployment, which is probably fine in this instance since the network 
barclamp can have only one deployment, and its name will most likely not be 
shown in the GUI as a result.  But, I think we have a bigger issue here in that 
translation is not working when running unit tests on Travis.  Unfortunately, I 
know very little about Travis or how translation is set up.

Can anyone help out here?


Thanks,

Chris
Dell

From: Dearborn, Chris
Sent: Wednesday, March 13, 2013 2:03 PM
To: Hirschfeld, Rob; ja...@suse.de; crowbar
Subject: RE: [Crowbar] [Still Failing] crowbar/travis-ci-crowbar#60 (master - 
ac51708)

Guys,

I'm looking at this issue right now, and the problem is not an assert 
statement.  Those were fixed in an earlier pull request.  I know what the issue 
is and am working on an email now.


Thanks,

Chris

From: crowbar-bounces On Behalf Of Hirschfeld, Rob
Sent: Wednesday, March 13, 2013 2:01 PM
To: ja...@suse.de; crowbar
Subject: Re: [Crowbar] [Still Failing] crowbar/travis-ci-crowbar#60 (master - 
ac51708)

Sigh  I'll correct.

Still working on the dev 1.9.3 migration over here.

From: crowbar-bounces On Behalf Of James Tan
Sent: Wednesday, March 13, 2013 12:54 PM
To: crowbar
Subject: Re: [Crowbar] [Still Failing] crowbar/travis-ci-crowbar#60 (master - 
ac51708)

On 03/13/2013 06:38 PM, Travis-CI wrote:
The build is still failing.


Repository

crowbar/travis-ci-crowbar

Build #60

https://travis-ci.org/crowbar/travis-ci-crowbar/builds/5477613


Anyone looking into this failure on Ruby 1.9.3? Details here:

https://travis-ci.org/crowbar/travis-ci-crowbar/jobs/5477618/#L562

Mostly an issue of incorrectly using "assert" when you really want to have 
"assert_equal". Please also run tests on Ruby 1.9.3, as Ruby 1.8.7 is going to 
be dropped by upstream in June.

Thanks,
James T.
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] testing and pull requests

[James Tan]

On 03/13/2013 11:32 AM, Adam Spiers wrote:
>>> "What testing do we run before submitting?" is a key question; thanks for
>>> > >bringing it up.  Unfortunately the answer is currently "only whatever we 
>>> > >run
>>> > >manually", since Travis only runs on a merged repository containing 
>>> > >master
>>> > >from each barclamp repository, due to barclamps currently being unable 
>>> > >to run
>>> > >their unit tests standalone.
>>> > >(There are Trello cards to address this, but it's unlikely to get fixed 
>>> > >soon.)
>> >
>> >The general pattern we've been following is to have barclamps extend 
>> >framework provided classes, and that code currently resides in 
>> >crowbar/crowbar_framework... so to get to a point where we can run unit 
>> >tests for standalone barclamps will require copying in the framework 
>> >(chicken, meet egg...).

The root of all evil here is really the tight coupling we have between 
the Crowbar framework itself and the barclamps. We really need to focus 
on decoupling them so that can be standalone.

> Hmm ... hopefully there is an industry-wide best practice for reusing
> shared code between engines which we can follow?

A gem? :-)

Cheers,
James T.

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] Rubygems 2.0 - breaks bundle and others

[Andi_Abes]

Well, thank you - Travis found it before I hit it ;)

From: James Tan [mailto:ja...@suse.de]
Sent: Wednesday, March 13, 2013 2:09 PM
To: Abes, Andi
Cc: crowbar
Subject: Re: [Crowbar] Rubygems 2.0 - breaks bundle and others

Hi Andi,

On 03/13/2013 01:14 PM, andi_a...@dell.com wrote:
If you're about to install a new build env, or building with internet wide 
downloads, keep reading. Otherwise it's safe to ignore.
The latest version of RubyGems (2.0.3) has a bunch of breaking changes, some 
break bundler, others break chef - lots of fun to be had.
The last safe version on the 1.8 train is 1.8.25. To get it:
gem update --system 1.8.25
more details here: http://blog.rubygems.org/2013/02/24/2.0.0-released.html

Thanks for the heads up. This saved me a lot of time!

Cheers,
James T.
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] Porting core cookbooks to attribute injection

[Judd Maltin]

Hi guys,

My blog entry has a lot of the design patterns, and I quote a bunch of
others.  It's a good backgrounder:
http://newgoliath.wordpress.com/2013/03/08/orchestration-consistency-and-community-cookbooks/

This past sprint I started the code updates.  This coming sprint I'll be
finishing them up, teamed up with Wayne from our team.

If we are to create a short list of the pattern, here it is:

* no attribute l-values in recipes
* no OS detection in recipes
* define all OS dependent variables in attibutes/default.rb
* define all l-values in attirubtes/default.rb
* do not use recipes/default.rb
* recipes should be named as if they were methods
* in recipes, oft-repeated patterns should be abstracted into a function in
a library/.rb
   * example:
https://github.com/att-cloud/cookbook-openstack-common/blob/master/libraries/endpoints.rb







On Wed, Mar 13, 2013 at 1:56 PM, James Tan  wrote:

> On 03/07/2013 11:13 AM, Vincent Untz wrote:
> > Hi,
> >
> > First, thanks for the overview on attribute injection in yesterday's
> > call. This was quite useful to better understand what this all means and
> > why we are moving this way, as well as how things can look like (with
> > the keystone cookbook example).
> >
> > Ideally, we'd have a good example of porting a cookbook of a core
> > barclamp to AI, which people can use as model when porting the other
> > cookbooks. Is anybody working on such an example?
>
> Agreed, a reference implementation would be very useful. Do we already
> have something like that? If not, let's pick one and make it so?
>
> Cheers,
> James T.
>
>
> ___
> Crowbar mailing list
> Crowbar@dell.com
> https://lists.us.dell.com/mailman/listinfo/crowbar
> For more information: http://crowbar.github.com/
>



-- 
Judd Maltin
T: 917-882-1270
F: 501-694-7809
what could possibly go wrong?
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] Porting core cookbooks to attribute injection

[Judd Maltin]

oops.. by l-values I mean r-values.


On Wed, Mar 13, 2013 at 2:50 PM, Judd Maltin  wrote:

> Hi guys,
>
> My blog entry has a lot of the design patterns, and I quote a bunch of
> others.  It's a good backgrounder:
> http://newgoliath.wordpress.com/2013/03/08/orchestration-consistency-and-community-cookbooks/
>
> This past sprint I started the code updates.  This coming sprint I'll be
> finishing them up, teamed up with Wayne from our team.
>
> If we are to create a short list of the pattern, here it is:
>
> * no attribute l-values in recipes
> * no OS detection in recipes
> * define all OS dependent variables in attibutes/default.rb
> * define all l-values in attirubtes/default.rb
> * do not use recipes/default.rb
> * recipes should be named as if they were methods
> * in recipes, oft-repeated patterns should be abstracted into a function
> in a library/.rb
>* example:
> https://github.com/att-cloud/cookbook-openstack-common/blob/master/libraries/endpoints.rb
>
>
>
>
>
>
>
> On Wed, Mar 13, 2013 at 1:56 PM, James Tan  wrote:
>
>> On 03/07/2013 11:13 AM, Vincent Untz wrote:
>> > Hi,
>> >
>> > First, thanks for the overview on attribute injection in yesterday's
>> > call. This was quite useful to better understand what this all means and
>> > why we are moving this way, as well as how things can look like (with
>> > the keystone cookbook example).
>> >
>> > Ideally, we'd have a good example of porting a cookbook of a core
>> > barclamp to AI, which people can use as model when porting the other
>> > cookbooks. Is anybody working on such an example?
>>
>> Agreed, a reference implementation would be very useful. Do we already
>> have something like that? If not, let's pick one and make it so?
>>
>> Cheers,
>> James T.
>>
>>
>> ___
>> Crowbar mailing list
>> Crowbar@dell.com
>> https://lists.us.dell.com/mailman/listinfo/crowbar
>> For more information: http://crowbar.github.com/
>>
>
>
>
> --
> Judd Maltin
> T: 917-882-1270
> F: 501-694-7809
> what could possibly go wrong?
>
>
>


-- 
Judd Maltin
T: 917-882-1270
F: 501-694-7809
what could possibly go wrong?
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] Porting core cookbooks to attribute injection

[Andi_Abes]

> -Original Message-
> From: crowbar-bounces On Behalf Of James Tan
> Sent: Wednesday, March 13, 2013 1:57 PM
> To: Vincent Untz
> Cc: crowbar
> Subject: Re: [Crowbar] Porting core cookbooks to attribute injection
>
> On 03/07/2013 11:13 AM, Vincent Untz wrote:
> > Hi,
> >
> > First, thanks for the overview on attribute injection in yesterday's
> > call. This was quite useful to better understand what this all means
> > and why we are moving this way, as well as how things can look like
> > (with the keystone cookbook example).
> >
> > Ideally, we'd have a good example of porting a cookbook of a core
> > barclamp to AI, which people can use as model when porting the other
> > cookbooks. Is anybody working on such an example?
>
> Agreed, a reference implementation would be very useful. Do we already have
> something like that? If not, let's pick one and make it so?
>

So we have some partial examples, but nothing that completes the whole story. 
But, rather than wait for a complete example - could be incremental on this?

You might recall that the main drivers were:
* remove crowbarisms (or at least make them optional) so we can better share 
cookooks
* limit search as much as possible.

The swift cookbook is somewhat of a good example for the first, but not the 
second. If you look at the default attributes in [1] and specifically [2], and 
compare it to [3].

There are a few ideas from the AttrInj discussion that are captured by this 
example, and a few that are missing (the second list):
1) Default attributes set by the cookbooks, and server as an "API" of 
attributes that the recipe requires.
2) Those default attributes don't require "crowbar-ism" - in fact you can run 
them with just chef.
3) The default attributes are overridden to refer to values referring 
attributes generated by crowbar (a more full example below).
4) library wrappers for access to some attributes [4]. Though in this case, 
it's not deployment wide (as openstack-common), but rather embedded in swift... 
which should probably change to look more like [5].


What's not manifested:
1) search replacement
2) some form of allowing for pluggability of providers for services - e.g. [6]
3) library to publish status/info back into the server (outside of ohai) , e.g. 
to indicate if non-fatal failures have occurred in some recipe [7]


A walk through example for overriding attributes:
In the default attributes (used outside of crowbar) the source of disks to be 
used in swift storage nodes, is the ohai info, and it has very basic filtering:

# expression to find a hash of possible disks to be used.
default[:swift][:disk_enum_expr]= 'node[:block_device]'
# expression accepting a k,v pair for evaluation. if expression returns true, 
then the disk will be used.
# by default, use any sdX or hdX that is not the first one (which will hold the 
OS).
default[:swift][:disk_test_expr]= 'k =~/sd[^a]/ or k=~/hd[^a]/'

When run in crowbar, that attribute is overridden (using a role override 
attribute, that are ""injected"" into the proposal configuration) to refer to 
data put on the node like so (from [3]).
"disk_enum_expr": "node[\"crowbar\"][\"disks\"]",
"disk_test_expr": "v[\"usage\"] == \"Storage\"",


Hope this setting the general ""tone"" if not quite a pattern of the place 
we're trying to march to.




[1] 
https://github.com/cloudedge/barclamp-swift/blob/master/chef/cookbooks/swift/attributes/default.rb
[2] 
https://github.com/cloudedge/barclamp-swift/blob/master/chef/cookbooks/swift/attributes/default.rb#L48
[3] 
https://github.com/cloudedge/barclamp-swift/blob/master/bc-template-swift.json#L27
[4] 
https://github.com/cloudedge/barclamp-swift/blob/master/chef/cookbooks/swift/recipes/proxy.rb#L23
[5] 
https://github.com/att-cloud/cookbook-openstack-common/blob/master/libraries/endpoints.rb
[6] 
https://github.com/rcbops-cookbooks/monitoring/blob/master/providers/procmon_monit.rb
[7] 
https://github.com/crowbar/barclamp-ipmi/blob/master/chef/cookbooks/ipmi/recipes/ipmi-configure.rb#L48







> Cheers,
> James T.
>
>
> ___
> Crowbar mailing list
> Crowbar@dell.com
> https://lists.us.dell.com/mailman/listinfo/crowbar
> For more information: http://crowbar.github.com/

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] removing non-engine files?

[Darrel O'Pry]

I have to second James here. I just came off a project where they insisted
on keeping historic stuff in the repo. It only served to confuse people
coming onboard who weren't already familiar with the project.  Its really
easy to make a branch in you own repository and switch back to it or view
it on github if you need it for reference.
 On Mar 13, 2013 2:04 PM, "James Tan"  wrote:

> On 03/13/2013 04:54 PM, wayne_al...@dell.com wrote:
> > I would like to keep the old versions around for quick reference and
> delete them as each barclamp is fully integrated into the new system.
>
> While I understand the convenience of keeping these files around during
> integration, it's really confusing for everyone else. Can you remove
> them and refer via the git history instead?
>
> Thanks,
> James T.
>
> ___
> Crowbar mailing list
> Crowbar@dell.com
> https://lists.us.dell.com/mailman/listinfo/crowbar
> For more information: http://crowbar.github.com/
>
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

Re: [Crowbar] removing non-engine files?

[Andi_Abes]

I think we're talking about a few days, not as an ongoing pattern. Surely we're 
living with much bigger warts than this one...;)


From: crowbar-bounces On Behalf Of Darrel O'Pry
Sent: Wednesday, March 13, 2013 3:32 PM
To: James Tan
Cc: crowbar; Allen, Wayne
Subject: Re: [Crowbar] removing non-engine files?


I have to second James here. I just came off a project where they insisted on 
keeping historic stuff in the repo. It only served to confuse people coming 
onboard who weren't already familiar with the project.  Its really easy to make 
a branch in you own repository and switch back to it or view it on github if 
you need it for reference.
On Mar 13, 2013 2:04 PM, "James Tan" mailto:ja...@suse.de>> 
wrote:
On 03/13/2013 04:54 PM, wayne_al...@dell.com wrote:
> I would like to keep the old versions around for quick reference and delete 
> them as each barclamp is fully integrated into the new system.

While I understand the convenience of keeping these files around during
integration, it's really confusing for everyone else. Can you remove
them and refer via the git history instead?

Thanks,
James T.

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/

[Crowbar] Documentation system now honors ###_ ordering

[Rob_Hirschfeld]

All,

Andi and I talked about this simple change and I've got a pending commit for 
it...

If you add a # prefix to your documentation.md files then it will honor that as 
the order for the document.

For example 10_introduction.md will be ordered as 10.  100_details.md will 
ordered as 000100.  This changes allows you to control ordering of 
documentation without having to mess with the meta data file.

Rob
__
Rob Hirschfeld
Distinguished Cloud Solution Architect
Dell | Cloud Edge, Data Center Solutions
cell +1 512 909-7219 blog robhirschfeld.com, twitter @zehicle
Please note, I am based in the CENTRAL (-6) time zone

___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/