Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Paul Voccio
Woah, seems I missed a lot by not being around email today. I was a bit confused at to why we would want to have nova trackif an account was being used by a reseller. In digging back through the blueprint associated with this, it seems the idea is for the operator (in this case Rackspace, but whoe

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Eric Day
On Mon, Feb 07, 2011 at 08:50:58PM -0500, Jay Pipes wrote: > Eric, you and I have a database background. I know you understand that this: Of course, but the first pair of queries is not as bad as a query for every entity ID returned, which was in one of the previous emails (the main thing I was tr

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Greg
On Feb 7, 2011, at 7:58 PM, Jay Pipes wrote: > I'm not saying the federated auth/CMDB/whatever service is not a good > idea nor that it will not work. I want people to understand and > acknowledge the tradeoffs involved. Acknowledged? :) [Hehe, damned programmers will never agree on nearly anyt

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Jay Pipes
On Mon, Feb 7, 2011 at 8:24 PM, Jorge Williams wrote: > On Feb 7, 2011, at 5:43 PM, Jay Pipes wrote: > >> What if I don't want to get "my" servers only? What if I want to list >> another organization's servers, and that organization's child >> organizations' servers? > > That sort of information a

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Jay Pipes
On Mon, Feb 7, 2011 at 7:05 PM, Eric Day wrote: > On Mon, Feb 07, 2011 at 06:43:37PM -0500, Jay Pipes wrote: >> For any of these types of requests, the process is always going to be this: >> >> request >>   -> api node >>      -> auth plugin get list of entities for some entity >>   <- api node, f

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Jorge Williams
On Feb 7, 2011, at 5:43 PM, Jay Pipes wrote: > What if I don't want to get "my" servers only? What if I want to list > another organization's servers, and that organization's child > organizations' servers? That sort of information and those sort of quires fall within the realm of configuration

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Monsyne Dragon
On 2/7/11 6:45 PM, Greg wrote: Perhaps thought should be given to other services besides Nova. Swift has no central accounts database. For it to gather "all accounts like x" type queries, it'd have to hit up every account node. Probably the same for a queue service. A database service might ha

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Greg
Perhaps thought should be given to other services besides Nova. Swift has no central accounts database. For it to gather "all accounts like x" type queries, it'd have to hit up every account node. Probably the same for a queue service. A database service might have a central db, and I guess Nova

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Eric Day
On Mon, Feb 07, 2011 at 06:43:37PM -0500, Jay Pipes wrote: > What if I don't want to get "my" servers only? What if I want to list > another organization's servers, and that organization's child > organizations' servers? I guess I'm thinking public cloud mostly, but sure, perhaps admin entities wa

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Jay Pipes
On Mon, Feb 7, 2011 at 4:38 PM, Eric Day wrote: > Hi Jay, > > When would the iterating query need to run? I want to make sure we > consider the various use cases. > > If you're thinking this would be from the billing/audit system, I don't > think it should not need to query back into nova. The eve

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread John Purrier
There are a lot of benefits to having an external system be responsible for handling usage data from Nova, Swift, or other OpenStack services. I would call out: 1. Simplification of code within each service. The collection and publication of usage data is "dumb". store the service usage d

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Eric Day
Hi Jay, When would the iterating query need to run? I want to make sure we consider the various use cases. If you're thinking this would be from the billing/audit system, I don't think it should not need to query back into nova. The events being pushed out should contain enough data to answer any

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Jay Pipes
OK, no worries, I'm just pointing out the issues so down the road it won't be a surprise... Keep in mind that if we go down this route, there won't be an efficient way to ask this: GET /servers/by-account-id/customerB/include_children=1 Instead, the outside software will have to be iterating acr

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Jorge Williams
Indeed, each Open Stack service (object-store, compute, ...) should only contain operational data and it should push audit events to external systems. Pushing billing/audit data outside of nova provides a flexible approach that allows operators to perform optimizations according to their own n

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Monsyne Dragon
On 2/7/11 2:49 PM, Eric Day wrote: Thanks for explaining things further, Jay. I agree if we want external systems poking into Nova for audit/billing queries, then yes, this gets inefficient. My assumption is that Nova specific DBs only contain operational data required for production and it woul

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Eric Day
Thanks for explaining things further, Jay. I agree if we want external systems poking into Nova for audit/billing queries, then yes, this gets inefficient. My assumption is that Nova specific DBs only contain operational data required for production and it would push billing/audit events to some e

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Jay Pipes
On Mon, Feb 7, 2011 at 2:46 PM, Eric Day wrote: > Perhaps we can be a bit more explicit about what the performance > issues will be. From your original email, you listed a few queries > with an example "X-Y-Z" string for the account ID. My answer to > those concerns is to let an account plugin par

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Eric Day
I apologize if any of my emails are coming off the wrong way, I'm just trying to explore options. I don't think anyone has decided anything yet, and I really want to address the concerns you have. Perhaps we can be a bit more explicit about what the performance issues will be. From your original e

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Jay Pipes
I give up trying to explain why I think this will lead to suboptimal performance and hacks all over the external ecosystem. If the decision is to punt and let the external "auth systems" be responsible for understanding the relationships between accounts and users in Nova, so be it. I'll just pre

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Eric Day
On Mon, Feb 07, 2011 at 08:50:28AM -0500, Jay Pipes wrote: > No, I think you've missed my point. Comments inline... Actually, I think I did get all your points, we're just not connecting somewhere. :) > On Mon, Feb 7, 2011 at 12:35 AM, Eric Day wrote: > > I disagree with your disagreement. :) >

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Monsyne Dragon
On 2/6/11 5:09 PM, Brian Schott wrote: Going back to the original question, it seems that projects has all of the necessary authentication, quota, audit hooks, and role-based management of granting users access to projects to support multi-tennant management, but the issue is that you can't ha

Re: [Openstack] Google Summer of Code 2011

2011-02-07 Thread Stephen Spector
My apologies, the link did not copy over correctly - http://wiki.openstack.org/Project%20List Thanks, On Feb 7, 2011, at 10:47 AM, Stephen Spector wrote: > Thanks for everyone's feedback to the idea of supporting GSOC 2011. I have > received emails from several people interested in becoming

[Openstack] Fwd: Pondering multi-tenant needs in nova.

2011-02-07 Thread Brian Schott
Ah, was member of Nova team, but not OpenStack team. I think this is still relevant to the discussion. I've read here that accounts are a 1) collection of projects, or 2) a collection of users, or 3) a billing / audit tag on usage reports and/or logs. The blueprint seems to indicate that the

Re: [Openstack] Google Summer of Code 2011

2011-02-07 Thread Stephen Spector
Thanks for everyone's feedback to the idea of supporting GSOC 2011. I have received emails from several people interested in becoming a mentor. As a next step, please list yourself on the GSOC 2011 Wiki page at http://wiki.openstack.org/Project List along with your project ideas/descriptions.

Re: [Openstack] Google Summer of Code 2011

2011-02-07 Thread Thierry Carrez
Rick Clark wrote: > In order to be successful, quite a bit of thought and planning needs to > be put into a GSOC project. I've seen other open source projects look > bad by not being organized enough. This needs to be way more put > together, and thought out before I would want to see openstack's

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Jay Pipes
Cool, thanks for the info, Greg :) On Mon, Feb 7, 2011 at 11:13 AM, Greg wrote: > On Feb 7, 2011, at 9:42 AM, Jay Pipes wrote: > >> OK, but can you head all the accounts under another account? Or is >> that done via log processing? > > There no concept of accounts under another account with Swift

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Greg
On Feb 7, 2011, at 9:42 AM, Jay Pipes wrote: > OK, but can you head all the accounts under another account? Or is > that done via log processing? There no concept of accounts under another account with Swift. An external system could do head requests however desired. But yeah, that sort of thing

Re: [Openstack] Google Summer of Code 2011

2011-02-07 Thread Rick Clark
In order to be successful, quite a bit of thought and planning needs to be put into a GSOC project. I've seen other open source projects look bad by not being organized enough. This needs to be way more put together, and thought out before I would want to see openstack's name on it. Rick sign

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Jay Pipes
On Mon, Feb 7, 2011 at 10:33 AM, Greg wrote: > On Feb 7, 2011, at 8:30 AM, Jay Pipes wrote: > >> What Swift APIs are available for a reseller to query which of its >> customer accounts have consumed X resources? Or does Swift punt and >> make the reseller calculate all those things? > > A bit of b

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Greg
On Feb 7, 2011, at 8:30 AM, Jay Pipes wrote: > What Swift APIs are available for a reseller to query which of its > customer accounts have consumed X resources? Or does Swift punt and > make the reseller calculate all those things? A bit of both. You can head an account to get get disk usage, obj

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Jay Pipes
On Mon, Feb 7, 2011 at 9:22 AM, Greg wrote: > If it's any help, Swift just uses an opaque account string. With Cloud Files > we have several resellers each with their own sets of how accounts and users > should be structured. We backed away quickly from all that and went the route > of "here's

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Greg
If it's any help, Swift just uses an opaque account string. With Cloud Files we have several resellers each with their own sets of how accounts and users should be structured. We backed away quickly from all that and went the route of "here's an account identifier value, it must be unique amongs

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Jay Pipes
No, I think you've missed my point. Comments inline... On Mon, Feb 7, 2011 at 12:35 AM, Eric Day wrote: > I disagree with your disagreement. :) > > When we have string based ID's like this, it doesn't need to translate > directly into a varchar column for operations. First, auth data may not > be

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Jay Pipes
I am really proposing to get rid of the projects table entirely and consume it with the SQL structure I proposed that can handle nested organizational units. On Sun, Feb 6, 2011 at 6:09 PM, Brian Schott wrote: > Going back to the original question, it seems that projects has all of the > necessa

Re: [Openstack] Multi Clusters in a Region ...

2011-02-07 Thread Sandy Walsh
Hi again, I've made some final changes to the Multi-Cluster spec and hope to start coding this week. In a nutshell: I spent the past week messing with RabbitMQ clusters, including WAN clusters between several Rackspace Data Centers. RabbitMQ doesn't really support inter-cluster communication

Re: [Openstack] Pondering multi-tenant needs in nova.

2011-02-07 Thread Sandy Walsh
+1 From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Eric Day [e...@oddments.org] Sent: Monday, February 07, 2011 1:35 AM To: Jay Pipes Cc: openstack@lists.

Re: [Openstack] Google Summer of Code 2011

2011-02-07 Thread Soren Hansen
I agree this is a good idea. I'd be happy to be a mentor. -- Soren Hansen Ubuntu Developer    http://www.ubuntu.com/ OpenStack Developer http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchp