[Openstack] Burrow Release and Call for Developers

2011-08-29 Thread Eric Day
Hi everyone, I released a new version of the Burrow message queue service today, tagged in the diablo-4 milestone. You can find it here: https://launchpad.net/burrow The work during diablo has focused on finishing the conversion of prototype to a real project and increasing test coverage, docs,

Re: [Openstack] tests location

2010-07-27 Thread Eric Day
After seeing Soren's arguments for, I could actually go either way. Putting them in and possibly installing them is not the traditional way, but I see the potential value Soren points out. For those wanting to see the discussion so far, check out: https://code.launchpad.net/~termie/nova/move_test

[Openstack] Architecture for Shared Components

2010-07-30 Thread Eric Day
Hi Everyone, A number of us have been discussing different ways of sharing components between different projects in OpenStack, such as auth*, caching, rate limiting, and so on. There are a few ways to approach this, and we thought it would be best to put this out on the mailing list for folks to d

Re: [Openstack] Architecture for Shared Components

2010-08-02 Thread Eric Day
Hi Jorge, Michael, Yeah, this is pretty much what I had in mind. Beyond having services that get called out from APIs, the implementation within the projects should not be specific as well. For example, there should be a generic auth API that can be used across projects, but this could be backed b

Re: [Openstack] Architecture for Shared Components

2010-08-02 Thread Eric Day
Hi Jorge, I think we may not be on the same page here. I can't speak for what Michael meant, but this is what I had in mind: http://oddments.org/tmp/os_module_arch.png This removes the intermediate proxies and instead relies on the scaling out of API endpoint and the services it uses. Different

Re: [Openstack] Architecture for Shared Components

2010-08-03 Thread Eric Day
Hi Michael, On Tue, Aug 03, 2010 at 10:06:02AM -0400, Michael Gundlach wrote: >But I think we're getting close :) All three of us have slightly >different approaches in mind, but we're narrowing down the areas where we >disagree. I've tried listing our agreements and disagreements be

Re: [Openstack] Architecture for Shared Components

2010-08-03 Thread Eric Day
Hey! On Tue, Aug 03, 2010 at 03:24:15PM -0400, Michael Gundlach wrote: >Hooray again for WSGI solving this problem :) In-process transformations >that can become HTTP proxies as needed! I have a prototype of a layered WSGI app that talks AMQP (via carrot module like in Nova) to Rabbit du

Re: [Openstack] Architecture for Shared Components

2010-08-03 Thread Eric Day
Hi Jorge, On Tue, Aug 03, 2010 at 05:34:59PM -0500, Jorge Williams wrote: >I'm not that familiar with WSGI but it looks like you're proposing >something similar to what I have above. > >WSGI may indeed be the solution, but I would say, what should be split >into a separate app sh

Re: [Openstack] Test Frameworks ...

2010-11-22 Thread Eric Day
On Mon, Nov 22, 2010 at 05:44:38PM -0500, Jay Pipes wrote: > On Mon, Nov 22, 2010 at 4:20 PM, Michael Gundlach > > I vote we also get rid of run_tests.sh and the unit test base class, so we > > aren't tempted to add dependencies required for nova tests to run.  Running > > tests then becomes typing

Re: [Openstack] What each Bug status should mean

2010-11-24 Thread Eric Day
On Wed, Nov 24, 2010 at 11:51:02AM +0100, Thierry Carrez wrote: > The first question is what "Fix committed" and "Fix released" should > mean. There are two possible directions: I'm good either way, I think both side are valid. > The second question revolves around the "Confirmed" and "Triaged" >

Re: [Openstack] Merge proposals and criteria for approval

2010-12-21 Thread Eric Day
I also agree, except for the last part. While you can break a test case but not a feature, we fail at providing adequate coverage for the feature (one of the criteria stated below). Not everyone will have the ability to verify functionality without test cases, so we should not depend on manual exte

Re: [Openstack] Use of IANA-registered ports

2011-01-02 Thread Eric Day
For production deployments, the default port should be 80, no? I imagine most production deployments will be running port 80 and have different sets of hosts running each service (swift, glance, nova). Four single-machine setup we should explain how to change the ports so they don't interfere, but

Re: [Openstack] Use of IANA-registered ports

2011-01-03 Thread Eric Day
rage API nodes with auth and proxy together (proxy > will use port 80, but we still need one for auth). > > For Nova, I think we're OK with the HTTP ports, because most of the > components are using rabbitmq for communication. For Glance, I'm not sure. > > Cheers, > >

Re: [Openstack] Glance/Nova Snapshot Changes

2011-01-03 Thread Eric Day
Is there a reason you need to even touch Glance from the Nova API server before returning metadata? Could you just generate and return what that will be, or do you need to obtain something from Glance for the return (like image_id)? For compute API/manager, you can make this a call instead of a ca

Re: [Openstack] Deprecating nova-objectstore

2011-01-17 Thread Eric Day
On Mon, Jan 17, 2011 at 03:53:44PM +, Ewan Mellor wrote: > We could do it in two steps. You could set up nova-compute -> Glance -> > nova-objectstore (testing purposes only). This would allow us to remove the > S3 code from Nova, but people could still use nova-objectstore if they don't >

Re: [Openstack] Google Summer of Code 2011

2011-02-04 Thread Eric Day
Having been a mentor before for other projects, I strongly agree that we should do this. It's a great way to get more folks involved in the project. -Eric On Fri, Feb 04, 2011 at 02:37:18PM -0600, Stephen Spector wrote: >OpenStack Developers: >Now that you have a few days off (just kiddin

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-06 Thread Eric Day
I disagree with your disagreement. :) When we have string based ID's like this, it doesn't need to translate directly into a varchar column for operations. First, auth data may not be stored as SQL at all for some systems and could be broken out into key/value pairs with some indexed. It could als

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Eric Day
On Mon, Feb 07, 2011 at 08:50:28AM -0500, Jay Pipes wrote: > No, I think you've missed my point. Comments inline... Actually, I think I did get all your points, we're just not connecting somewhere. :) > On Mon, Feb 7, 2011 at 12:35 AM, Eric Day wrote: > > I disagree

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Eric Day
al "auth systems" be > responsible for understanding the relationships between accounts and > users in Nova, so be it. > > I'll just predict the performance issues up front and call it a day. > > Cheers, > jay > > On Mon, Feb 7, 2011 at 12:46 PM, Eric Day wrote:

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Eric Day
e. :) My vote would still be pushing events to an OpenStack firehose and let billing/audit tap into that, but of course I'll defer to the group decision. -Eric On Mon, Feb 07, 2011 at 03:22:06PM -0500, Jay Pipes wrote: > On Mon, Feb 7, 2011 at 2:46 PM, Eric Day wrote: > > Perhaps

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Eric Day
ons that > > directly effect how our operators do business.  It also means that > > different service teams (swift, nova) can plug in a very similar manner, > > without having to replicate a lot of code. > > > > -jOrGe W. > > > > On Feb 7, 2011,

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Eric Day
On Mon, Feb 07, 2011 at 06:43:37PM -0500, Jay Pipes wrote: > What if I don't want to get "my" servers only? What if I want to list > another organization's servers, and that organization's child > organizations' servers? I guess I'm thinking public cloud mostly, but sure, perhaps admin entities wa

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Eric Day
On Mon, Feb 07, 2011 at 08:50:58PM -0500, Jay Pipes wrote: > Eric, you and I have a database background. I know you understand that this: Of course, but the first pair of queries is not as bad as a query for every entity ID returned, which was in one of the previous emails (the main thing I was tr

Re: [Openstack] Multi Clusters in a Region ...

2011-02-09 Thread Eric Day
Hi Sandy, Replying via email since it seems not much discussion is happening via the etherpad. I'm glad to see we're going with REST-API for inter-zone communication. This API should be the same thing we use for connecting any two clouds together, for example a customers private cloud and a publi

[Openstack] Proposed OpenStack Service Requirements

2011-02-09 Thread Eric Day
The email thread with the subject "[RFC] OpenStack Programming Model Framework" from Jan 3rd covered a few ground proposals for OpenStack projects, mainly with a focus on API. I'd like to expand on this a bit more. I think everyone in is in agreement that each service should default to a REST API

Re: [Openstack] Multi Clusters in a Region ...

2011-02-09 Thread Eric Day
Hi Jay, On Wed, Feb 09, 2011 at 07:05:14PM -0500, Jay Pipes wrote: > I'm not sure I understand what the benefits of naming a zone as a URI > as opposed to just some name that describes it ("USNORTH", > "WINDOWSCAPABLE", "HIGH_SLA", whatever) Those names feel more like attributes of a zone to me,

Re: [Openstack] Proposed OpenStack Service Requirements

2011-02-10 Thread Eric Day
On Wed, Feb 09, 2011 at 08:00:02PM -0500, Jay Pipes wrote: > > There is other common functionality we should consider for OpenStack > > services. We don't need anything too formal, just a "best practices" > > document that can change with time. This will hopefully also drive > > openstack-common pr

Re: [Openstack] Multi Clusters in a Region ...

2011-02-10 Thread Eric Day
since it's more > of a scheduler issue, but I think it's a good idea. > > Keep it coming! > -Sandy > > > _ > > ___ > From: Eric Day [e...@oddments.org] > Sent: Wednesday, February 09, 2011 1:17 PM > To:

Re: [Openstack] Allowing clients to pass capability requests through tags?

2011-02-11 Thread Eric Day
The main reason I was proposing full location/zone of objects is to allow this type of 'near' scheduling to happen without understanding what the actual object is. For example, imagine we want to start an instance near a particular swift object. We could query the swift object and in the metadata t

Re: [Openstack] Allowing clients to pass capability requests through tags?

2011-02-11 Thread Eric Day
ially if it means > enforcing a rigid hierarchy. > > $0.02 > > -S > > > > > From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net > [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf

Re: [Openstack] xen server agent code in nova?

2011-02-11 Thread Eric Day
We can always start this as part of nova for developer convenience until the guest agent APIs stabilize. It's trivial to break this out into another project later on once other options start to emerge. -Eric On Fri, Feb 11, 2011 at 06:51:40PM +, Chris Behrens wrote: > > On Feb 11, 2011, at 5

Re: [Openstack] Allowing clients to pass capability requests through tags?

2011-02-11 Thread Eric Day
duler/zone mapping. For example, a volume may return zone '87af1b7c', which internally all services could lookup and translate to where that maps in the deployment hierarchy. I was hoping DNS names were abstract enough to reduce complexity, but perhaps they are not. :) -Eric >On Feb

Re: [Openstack] Allowing clients to pass capability requests through tags?

2011-02-11 Thread Eric Day
an admin api. I > don't think high level zone info is enough information for the scheduler > to decide where the instance should go. Especially with things like swift, > where there are three copies that may be in different zones altogether. > > Vish > On Feb 1

Re: [Openstack] Allowing clients to pass capability requests through tags?

2011-02-11 Thread Eric Day
n fallback to course zones for > objects > that don't need it. > > Vish > > On Feb 11, 2011, at 12:22 PM, Eric Day wrote: > > > If we go the admin-api lookup route, I worry about this as the number > > of services grow. If nova needs to support launching inst

[Openstack] Queue Service

2011-02-14 Thread Eric Day
Hi everyone, When looking at other services to include as part of OpenStack, the first that comes to mind for many is a queue. A queue service can not only provide a useful public cloud service, but can also provide one of the building blocks for other services. I've been leading an effort to rese

Re: [Openstack] Proposed OpenStack Service Requirements

2011-02-14 Thread Eric Day
On Sun, Feb 13, 2011 at 01:37:29PM -0500, Todd Willey wrote: > On Wed, Feb 9, 2011 at 1:38 PM, Eric Day wrote: > > * Firehouse - So far we've not discussed this too much, but I > >  think when we did there was agreement that we need it. As more > >  service come int

Re: [Openstack] Queue Service

2011-02-14 Thread Eric Day
icated machines more appropriate for the task). Async queues provide specialization of tasks, help lower response time, and make horizontal scale-out much easier, among other things. -Eric On Mon, Feb 14, 2011 at 11:20:41AM -0700, Pete Zaitcev wrote: > On Mon, 14 Feb 2011 09:51:42 -0800 > Er

Re: [Openstack] Queue Service

2011-02-14 Thread Eric Day
with the API email, I also have an initial roadmap email of features so we can add and prioritize various queue features (like this) with various milestones. I want to start very simple and work incrementally, but also want to make sure we fully understand these and other advanced ideas so we can pla

Re: [Openstack] Queue Service

2011-02-14 Thread Eric Day
mples. Please let me know if any clarification is needed and I'll update the wiki. Feedback and discussion on use cases and what you think the service should look like is very appreciated. Thanks, -Eric On Mon, Feb 14, 2011 at 09:51:42AM -0800, Eric Day wrote: > Hi everyone, > > Wh

Re: [Openstack] Queue Service

2011-02-14 Thread Eric Day
wrote: > Eric, > > Just looking at the http operations. Shouldn't the calls be around the > account then the queue? > > GET /$account_id/queue/id to list all the queues > GET /$account_id/queue/id/message/id > > Am I off here? Thoughts? > > pvo > &

Re: [Openstack] Queue Service

2011-02-14 Thread Eric Day
them, they just exist when there is an active message in it. -Eric > On 2/14/11 5:54 PM, "Eric Day" wrote: > > >Yeah, for anonymous access that would be the case. Those are not needed > >when the request comes in with OpenStack Auth headers (like nova). > > >

Re: [Openstack] Contribute to the PyCon talk on OpenStack!

2011-02-15 Thread Eric Day
Hi Ed, I think there is plenty of material on the mailing list/blogs/... to create a great talk from if you want to keep it focused on twisted vs eventlet vs other options for highly scalable systems (both in a technical and social way). That is where I would start and then perhaps start asking fo

Re: [Openstack] Queue Service

2011-02-15 Thread Eric Day
Hi Muriel, On Tue, Feb 15, 2011 at 10:04:20AM +0100, Muriel wrote: > i'm completely new to openstack but i started working on something > similar to eucalyptus. In the coming months we will begin a project > to bring our work on openstack and share it with you. Cool, looking forward to it! > The

Re: [Openstack] Queue Service

2011-02-15 Thread Eric Day
cept: application/xml > >> X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cb > >> > >> > >> > >> This seems a bit different than what you're suggesting. What am I missing? > >> Shouldn't the account id be in the request with the au

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Eric Day
Hi Sandy, On Wed, Feb 16, 2011 at 06:19:52PM +, Sandy Walsh wrote: > Hmm. You wouldn't really need to re-marshall the request. Just > copy the needed headers & url, and pass along the body as you received > it. Basically you are just > acting as a sort of http proxy.

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Eric Day
On Wed, Feb 16, 2011 at 01:02:22PM -0500, Jay Pipes wrote: > >> [Sorry, yes the instance name is passed in on the request, but the > >> instance ID is what's needed (assuming of course instance ID is unique > >> across zones.)] > > > >        The ID is determined early in the process; well before

Re: [Openstack] Queue Service

2011-02-16 Thread Eric Day
On Tue, Feb 15, 2011 at 01:18:46PM -0800, David Strauss wrote: > Regardless of whether the system uses AMQP, I am a fan of AMQP's > flexible routing model. Please don't build a service that lacks fanout > and other non-1:1 distribution patterns. Non-1:1 routing is useful for > web cache invalidatio

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Eric Day
On Wed, Feb 16, 2011 at 03:59:10PM -0500, Jay Pipes wrote: > But, as I mentioned to Sandy on IRC, caching and performance should be > a secondary concern. The primary concern, right now, is just making > this all work. In other words, getting multiple zones up and running, > each knowing about thei

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Eric Day
On Wed, Feb 16, 2011 at 04:33:06PM -0500, Jay Pipes wrote: > > While I would agree with this most of the time, there are some cases > > where "optimizing later" just doesn't work. Or, "optimizing" means > > rewriting everything you've done and replacing it with something > > that will scale seamles

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Eric Day
Good points Soren, and this is why I was suggesting we not solve this problem with a cache, but instead an eventually consistent replication stream of the aggregate data. -Eric On Wed, Feb 16, 2011 at 11:12:50PM +0100, Soren Hansen wrote: > 2011/2/16 Ed Leafe : > > This was one of the issues we d

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Eric Day
On Wed, Feb 16, 2011 at 11:20:31PM +0100, Soren Hansen wrote: > 2011/2/16 Sandy Walsh : > >> Hmmm... I am not sure about exposing internal structure to customers in > >> this > >> way.  Would you really want the more 'internal' zones exposed? > > To Jay's point, the "control panel" would hide all

[Openstack] Queue Service, next steps

2011-02-17 Thread Eric Day
Thanks to everyone who gave great feedback on the first queue service thread. I've updated the wiki page to include the suggestions. http://wiki.openstack.org/QueueService With a decent vision of what we want to build, the next step is figuring out how. In a previous thread it was suggested that

Re: [Openstack] Queue Service, next steps

2011-02-17 Thread Eric Day
Carlen >wrote: > > I'll put in a +1 for Erlang as an OpenStack supported platform. We'd be > able to write a stable queue with a much more concise code base, and > this is project would be a great fit for Erlang. > Devin > On Feb 17, 2011, at 2:21

Re: [Openstack] Queue Service, next steps

2011-02-18 Thread Eric Day
nt programming is much less error prone. -Eric > > >Personally, I'd prefer C++ since that's what I'm used to, but I'd be open to > >learning Erlang, too. Been wanting to learn it for a while now. > > >-jay > > On Fri, Feb 18, 2011 at 1:1

Re: [Openstack] Queue Service, next steps

2011-02-18 Thread Eric Day
we can get a group together. > > On 2/18/11 4:03 AM, "Curtis Carter" ; > wrote: > > >I'm in if it's done in Erlang. > >I am willing to give talks or help anyone wanting to learn Erlang in > the > >Austin/San Antonio

Re: [Openstack] Queue Service, next steps

2011-02-18 Thread Eric Day
On Fri, Feb 18, 2011 at 11:20:23AM -0600, Monsyne Dragon wrote: > Secondly, could we leverage existing opensource codebases to > accomplish this? I know AMQP was having issues over WAN links, and > likely isn't suited for disconnected operations (I've seen one > multitenant hosted queueing servic

Re: [Openstack] Queue Service, next steps

2011-02-18 Thread Eric Day
Yes, and with whatever language we choose, this will be the case. We need to write the "kernel" and provide plugin interfaces to write things like backing stores for any number of existing storage solutions, public protocol modules that may leverage existing libraries, auth modules that

Re: [Openstack] Queue Service, next steps

2011-02-18 Thread Eric Day
t for performance as this system has > > proven to beat many queue systems I tested written in c++. > > > > Thank You, > > Philip Schwartz > > Software Engineering > > LexisNexis RS > > O - 561 999 4472 > > C - 954 290 4024 > > > > -

Re: [Openstack] Queue Service, next steps

2011-02-18 Thread Eric Day
Hi Mike, On Fri, Feb 18, 2011 at 06:02:52PM -0600, Michael Barton wrote: > Maybe I'm describing a separate project, but a fault tolerant and > scalable queue would be more interesting to me than something like > RabbitMQ with a REST interface. There don't seem to be any reasonable > open-source i

Re: [Openstack] Queue Service, next steps

2011-02-18 Thread Eric Day
wrote: > On Fri, Feb 18, 2011 at 6:23 PM, Eric Day wrote: > > Perhaps I've been assuming some things, but I thought everyone > > understood that is what we are looking to build (fault-tolerant, > > horizontally scalable, ...). We're certainly not looking to build >

Re: [Openstack] Queue Service, next steps

2011-02-19 Thread Eric Day
Hi Mark, On Sat, Feb 19, 2011 at 11:18:26AM -0500, Mark Washenberger wrote: > It seems like put and post on ../queue[/id] are reversed from the usual > sense. I am probably just not familiar with the idioms for previous queue > services, but I imagine the following scheme. > >  POST .../queue

Re: [Openstack] Queue Service, next steps

2011-02-19 Thread Eric Day
worker was able to see them, at least until the hide value expires. -Eric On Sat, Feb 19, 2011 at 02:13:49PM -0500, Mark Washenberger wrote: > Can you give me some examples of the kinds of "update all messages in a > queue" requests you're thinking of? > > Thanks, >

Re: [Openstack] Queue Service, next steps

2011-02-19 Thread Eric Day
Hi Todd, That's the multicast example, for a normal 1-1 queue, look at the first example: Worker: POST /account/queue?wait=60&hide=60&detail=all (long-polling worker, request blocks until a message is ready) Client: PUT /account/queue (message inserted, returns unique id that was created) Wor

Re: [Openstack] Queue Service, next steps

2011-02-19 Thread Eric Day
Hi Mike, On Sat, Feb 19, 2011 at 07:42:12PM -0600, Michael Barton wrote: > On Fri, Feb 18, 2011 at 9:38 PM, Eric Day wrote: > > Hi Mike, > > > > You make a good point, I apologize for not documenting some of the > > ideas sooner. The architecture I had in mind b

Re: [Openstack] Queue Service, next steps

2011-02-21 Thread Eric Day
On Mon, Feb 21, 2011 at 06:46:58PM +, Chris Behrens wrote: > Do we have enough Erlang knowledge right now? I would not want to see Eric > get burned out because he can't get help. I've seen it happen before when > Erlang was chosen and the project ended up a complete failure. Knowing the i

Re: [Openstack] Queue Service, next steps

2011-02-21 Thread Eric Day
suggest we stick with C/C++. > > John > > -Original Message- > From: openstack-bounces+john=openstack@lists.launchpad.net > [mailto:openstack-bounces+john=openstack@lists.launchpad.net] On Behalf > Of Tim Bell > Sent: Monday, February 21, 2011 2:08 PM >

[Openstack] Queue service prototype

2011-02-21 Thread Eric Day
I decided to implement the current queue service specification in Python this weekend as it is described on the wiki page. It is functionally complete minus persistent queues and account verification (plugin points for the real implementation). You can find the links and description here (as well a

Re: [Openstack] Queue service prototype

2011-02-22 Thread Eric Day
Hehe. :) I was putting together a sample client with tests and before I knew it the whole server was written too. I agree now that this helps in the process, so Andy and the other folks advocating for a prototype were correct. -Eric > Vish > > On Feb 21, 2011, at 11:04 PM, Eric Day wrot

[Openstack] Burrow (queue service)

2011-02-22 Thread Eric Day
Hi everyone, I'd like to thank every who put in their opinion on the various queue service threads over the past couple weeks. It seems Erlang had the most enthusiasm from the language discussions, so we'll be going with that. We'll be able to move quickly with Erlang, which also means we'll be ab

Re: [Openstack] Novatools ...

2011-02-24 Thread Eric Day
For copyright headers, just add a new "Copyright 2011 OpenStack, LLC." line for existing files under the old copyright line. You can add a new license for new code for existing files, but that gets messy. For new files, just do as we usually do for new files (copyright + license brief). You can als

Re: [Openstack] Novatools ...

2011-02-24 Thread Eric Day
In regards to openstack tools, we certainly have some options. We could do everything from one big package with all tools for all languages/services to one project for each language/service (and all permutations in between). IMHO, I think it makes the most sense to keep the client tools for all (or

Re: [Openstack] Novatools ...

2011-02-24 Thread Eric Day
I would encourage using all lowercase for command line tools (oscompute), I don't really care what the name is though. :) -Eric On Thu, Feb 24, 2011 at 05:42:56PM +, Sandy Walsh wrote: >Perfect. >Objections? (naming bun-fights discouraged ;) >-S > > -

Re: [Openstack] Burrow (queue service)

2011-02-24 Thread Eric Day
Hi Greg, I've updated the wiki with a number of items you mention below, as well as some responses inline. I'm CC'ing the openstack list so others can see as well. On Wed, Feb 23, 2011 at 10:49:31AM -0600, gregory_alth...@dell.com wrote: > 1. What are the functional constraints of the queue? In

Re: [Openstack] Novatools ...

2011-02-24 Thread Eric Day
s there a concern that if I want to simply deploy the client tools on a >non-server, I have to get the full branch? Or did I miss a subtle point in >there? >-S > > From: Eric Day [e...@oddments.org] > > In regards to openstack tools, we certainly have

Re: [Openstack] Should the OpenStack API re-use the EC2 credentials?

2011-02-24 Thread Eric Day
I agree. I propose we always keep lp:nova (or lp:) stable, and instead create "trunks" like lp:nova/testing that all the test/regression systems can be run against. This is pretty similar to how we did things with Drizzle, a commit would bounce down the line, finlly landing in lp: when it was verif

Re: [Openstack] Novatools ...

2011-02-24 Thread Eric Day
r the full text. -Eric On Thu, Feb 24, 2011 at 08:20:16PM +0100, Thierry Carrez wrote: > Eric Day wrote: > > For copyright headers, just add a new "Copyright 2011 OpenStack, > > LLC." line for existing files under the old copyright line. You can add > > a new lic

Re: [Openstack] Novatools ...

2011-02-24 Thread Eric Day
ervices should be done through the OpenStack API, not via multiple > little tools... > > Just my 2 cents, > jay > > > John > > > > -Original Message- > > From: openstack-bounces+john=openstack@lists.launchpad.net > > [mailto:openstack-bounces+jo

Re: [Openstack] Decoupling of Network and Compute services for the new Network Service design

2011-02-24 Thread Eric Day
I agree with Vish, I think the correct approach is 3. I have some ideas on terminology and how to think about this. A "scheduler" should not be it's own top-level service. It should instead be a plugin point (more like auth or db). It would plug into new service called "kernel". Another way to look

Re: [Openstack] Burrow (queue service)

2011-02-24 Thread Eric Day
Hi Sandy, On Thu, Feb 24, 2011 at 07:42:34PM +, Sandy Walsh wrote: > Looks like a fun project Eric. I only got caught up on the ML this weekend > and I'm behind again already. It's a never-ending battle. I find routing all messages from jaypipes into /dev/null helps. :) > 1. Will broadcast

Re: [Openstack] Novatools ...

2011-02-24 Thread Eric Day
On Thu, Feb 24, 2011 at 03:48:25PM -0500, Jay Pipes wrote: > I just don't want to end up with: > > os-describe-images > os-describe-image-attribute > os-describe-instances > os-describe-groups > os-describe-zones > os-describe-keypairs > os-describe-volumes > os-describe-snapshots > > The above i

Re: [Openstack] Should the OpenStack API re-use the EC2 credentials?

2011-02-24 Thread Eric Day
The extra branches are just an implementation detail, we can have them or not. It's really a matter of if it's possible and/or easier to have jenkins fire off new jobs with arbitrary branches that need to be merged with trunk for each job vs merging and pushing to a staging branch and have the jobs

Re: [Openstack] Novatools ...

2011-02-24 Thread Eric Day
++ On Thu, Feb 24, 2011 at 02:33:42PM -0800, Devin Carlen wrote: > This is a bit nitpicky but I'd rather see it called just "nova", as in: > > nova describe images > > Who has strong opinions? > > On Feb 24, 2011, at 1:30 PM, Jay Pipes wrote: > > &

Re: [Openstack] Should the OpenStack API re-use the EC2 credentials?

2011-02-24 Thread Eric Day
of automated systems? > >On Thu, Feb 24, 2011 at 4:33 PM, Eric Day wrote: > > The extra branches are just an implementation detail, we can have > them or not. It's really a matter of if it's possible and/or easier > to have jenkins fire off n

Re: [Openstack] Should the OpenStack API re-use the EC2 credentials?

2011-02-24 Thread Eric Day
definitely be safer. > >On Thu, Feb 24, 2011 at 4:41 PM, Eric Day wrote: > > Yup. Right now when a -core member clicks 'Approve' > after code review, tarmac picks it up and pushes to trunk assuming > unittests pass. Instead it could push to staging a

Re: [Openstack] Should the OpenStack API re-use the EC2 credentials?

2011-02-24 Thread Eric Day
ly on different > branches. >On Thu, Feb 24, 2011 at 4:55 PM, Eric Day wrote: > > That is how it works now, but if we need to run tests that are not > on the tarmac machine, we need to push that local branch somewhere > so other things can test the same

Re: [Openstack] Novatools ...

2011-02-24 Thread Eric Day
aunchpad.net; John Purrier; Rick Clark > Subject: Re: [Openstack] Novatools ... > > This is a bit nitpicky but I'd rather see it called just "nova", as in: > > nova describe images > > Who has strong opinions? > > On Feb 24, 2011, at 1:30 PM, Jay Pipes wr

Re: [Openstack] Decoupling of Network and Compute services for the new Network Service design

2011-02-25 Thread Eric Day
time=0) or after a long outage (time=last update before going down). -Eric On Fri, Feb 25, 2011 at 08:27:24AM -0600, Ed Leafe wrote: > On Feb 24, 2011, at 2:02 PM, Eric Day wrote: > > > I agree with Vish, I think the correct approach is 3. I have some > > ideas on terminology

Re: [Openstack] Queue Service, next steps

2011-02-27 Thread Eric Day
Hi Raphael, On Sun, Feb 27, 2011 at 11:18:35AM +, Raphael Cohn wrote: >OpenStack's QueueService seems very interesting. As we have an existing >message queue implementation, we'd be happy to help you guys out. We're >about making messaging cloud-scale, so that everyone benefits. T

Re: [Openstack] Queue Service, next steps

2011-02-28 Thread Eric Day
Hi Raphael, On Mon, Feb 28, 2011 at 10:01:55AM +, Raphael Cohn wrote: >AMQP Observations >Your comments about AMQP seem to mostly be appropriate for one of the >older versions, eg 0-8, and I don't think they particularly apply to later >versions, eg 1-0. AMQP 0-8 did have some

Re: [Openstack] server affinity

2011-02-28 Thread Eric Day
Hi Gabe, There has been a lot of discussion about this, along with zone naming, structure, and so forth. I was propsing we not only make it part of Nova, but suggest all projects use the same locality zone names/tags to ensure cross-project locality. So, "yes", and don't make it nova-specific. :)

Re: [Openstack] server affinity

2011-02-28 Thread Eric Day
or > > example, in case for some performance reasons you wanted two servers with > > as few network hops as possible. If that still lines up with what you are > > talking about, great. > > > > Sorry about that! > > > > Gabe > > > > On Mon

[Openstack] State of OpenStack Auth

2011-03-01 Thread Eric Day
Hi everyone, I thought I'd take a moment to summarize the state of auth in the various OpenStack projects. With the recent discussion of OpenStack API auth in nova (bug+revert), how to structure accounts/users/projects/etc., and with Glance and Burrow needing auth solutions, now seems like a good

Re: [Openstack] State of OpenStack Auth

2011-03-01 Thread Eric Day
On Tue, Mar 01, 2011 at 09:47:00PM +0100, Soren Hansen wrote: > 2011/3/1 Eric Day : > > Signature based auth such as EC2 should also always require > > a secure channel too, but if not attacks are less severe since they > > are limited to reply attacks only (the request an

Re: [Openstack] State of OpenStack Auth

2011-03-01 Thread Eric Day
Well, hopefully with a shared, modular service, we can add a token module that uses cookies instead. :) -Eric On Tue, Mar 01, 2011 at 09:53:36PM +0100, Soren Hansen wrote: > On a subject of authentication, I've always been puzzled why the token > isn't just set as a standard http cookie? > > If

Re: [Openstack] State of OpenStack Auth

2011-03-01 Thread Eric Day
On Tue, Mar 01, 2011 at 10:02:37PM +0100, Soren Hansen wrote: > 2011/3/1 Eric Day : > > Well, hopefully with a shared, modular service, we can add a token > > module that uses cookies instead. :) > > Sure :) I was just hoping to extract whatever wisdom might have been >

[Openstack] Entities in OpenStack Auth

2011-03-01 Thread Eric Day
Hi everyone, I'd like to build off the last auth thread about where the various auth components are today and now look at what types of entities they manage. Right now only Swift and Nova have entities, so only those will be mentioned. Swift Swift has the concept of accounts, users, and groups.

Re: [Openstack] Entities in OpenStack Auth

2011-03-01 Thread Eric Day
On Tue, Mar 01, 2011 at 06:46:21PM -0600, Monsyne Dragon wrote: > 1) Break CloudServers API compatibility (a total no-no)? > and > >No. The value is added to the server management url that is reported when >you login. This is how the current Rackspace cloudservers API handles >

Re: [Openstack] Entities in OpenStack Auth

2011-03-01 Thread Eric Day
Hi Justin, On Tue, Mar 01, 2011 at 05:14:42PM -0800, Justin Santa Barbara wrote: >However, what I don't understand is how I can query my servers in project1 >and project2 (but not those in project3). *The only way I could see is >doing something like this: >*nova.openstack.org/v1.1

Re: [Openstack] Entities in OpenStack Auth

2011-03-01 Thread Eric Day
e.g. my X-Auth-Token could look like: "justinsb >project1,project2,project3 5OPr9UR2xk32K9ArAjO562e" (i.e. my username, >projects and a crypto signature) > Justin > >On Tue, Mar 1, 2011 at 5:51 PM, Eric Day wrote: > > Hi Justin, >

Re: [Openstack] server affinity

2011-03-02 Thread Eric Day
he approach > > really > > isn't that different from AWS in that essentially as an operator you use a > > prefix > > to separate your metadata from customer metadata...the prefix is simply > > defined by

  1   2   >