On 12/18/2013 12:27 PM, Tim Simpson wrote:
Please provide proof of that assumption or at least a general hypothesis that 
we can test.
I can't prove that the new agent will be larger as it doesn't exist yet.

Since nothing was agreed upon anyway, I don't know how you came to that 
conclusion.  I would suggest that any agent framework be held to an extremely 
high standard for footprint for this very reason.
Sorry, I formed a conclusion based on what I'd read so far. There has been talk 
to add Salt to this Unified Agent along with several other things. So I think 
its a valid concern to state that making this thing small is not as high on the 
list of priorities as adding extra functionality.

The C++ agent is just over 3 megabytes of real memory and takes up less than 30 
megabytes  of virtual memory. I don't think an agent has to be *that* small. 
However it won't get near that small unless making it tiny is made a priority, 
and I'm skeptical that's possible while also deciding an agent will be capable 
of interacting with all major OpenStack projects as well as Salt.

Nobody has suggested writing an agent that does everything.
Steven Dake just said:

"A unified agent addresses the downstream viewpoint well, which is 'There is only 
one agent to package and maintain, and it supports all the integrated OpenStack Program 
projects'."

So it sounds like some people are saying there will only be one. Or that it is 
at least an idea.

If Trove's communication method is in fact superior to all others, then perhaps 
we should discuss using that in the unified agent framework.
My point is every project should communicate to an agent in its own interface, 
which can be swapped out for whatever implementations people need.

  In fact I've specifically been arguing to keep it focused on facilitating 
guest<->service communication and limiting its in-guest capabilities to 
narrowly focused tasks.
I like this idea better than creating one agent to rule them all, but I would 
like to avoid forcing a single method of communicating between agents.

Also I'd certainly be interested in hearing about whether or not you think the 
C++ agent could made generic enough for any project to use.
I certainly believe much of the code could be reused for other projects. Right 
now it communicates over RabbitMQ, Oslo RPC style, so I'm not sure how much it 
will fall in line with what the Unified Agent group wants. However, I would 
love to talk more about this. So far my experience has been that no one wants 
to pursue using / developing an agent that was written in C++.

Thanks,

Tim
________________________________________
From: Clint Byrum [cl...@fewbar.com]
Sent: Wednesday, December 18, 2013 11:36 AM
To: openstack-dev
Subject: Re: [openstack-dev] [trove] My thoughts on the Unified Guest Agent

Excerpts from Tim Simpson's message of 2013-12-18 07:34:14 -0800:
I've been following the Unified Agent mailing list thread for awhile
now and, as someone who has written a fair amount of code for both of
the two existing Trove agents, thought I should give my opinion about
it. I like the idea of a unified agent, but believe that forcing Trove
to adopt this agent for use as its by default will stifle innovation
and harm the project.

"Them's fightin words". ;)

That is a very strong position to take. So I am going to hold your
statements of facts and assumptions to a very high standard below.

There are reasons Trove has more than one agent currently. While
everyone knows about the "Reference Agent" written in Python, Rackspace
uses a different agent written in C++ because it takes up less memory. The
concerns which led to the C++ agent would not be addressed by a unified
agent, which if anything would be larger than the Reference Agent is
currently.

"Would be larger..." - Please provide proof of that assumption or at least
a general hypothesis that we can test. Since nothing was agreed upon
anyway, I don't know how you came to that conclusion. I would suggest
that any agent framework be held to an extremely high standard for
footprint for this very reason.

I also believe a unified agent represents the wrong approach
philosophically. An agent by design needs to be lightweight, capable
of doing exactly what it needs to and no more. This is especially true
for a project like Trove whose goal is to not to provide overly general
PAAS capabilities but simply installation and maintenance of different
datastores. Currently, the Trove daemons handle most logic and leave
the agents themselves to do relatively little. This takes some effort
as many of the first iterations of Trove features have too much logic
put into the guest agents. However through perseverance the subsequent
designs are usually cleaner and simpler to follow. A community approved,
"do everything" agent would endorse the wrong balance and lead to
developers piling up logic on the guest side. Over time, features would
become dependent on the Unified Agent, making it impossible to run or
even contemplate light-weight agents.

Nobody has suggested writing an agent that does everything. A
framework for agents to build on is what has been suggested. In fact
I've specifically been arguing to keep it focused on facilitating
guest<->service communication and limiting its in-guest capabilities to
narrowly focused tasks.

Trove's interface to agents today is fairly loose and could stand to be
made stricter. However, it is flexible and works well enough. Essentially,
the duck typed interface of the trove.guestagent.api.API class is used
to send messages, and Trove conductor is used to receive them at which
point it updates the database. Because both of these components can be
swapped out if necessary, the code could support the Unified Agent when
it appears as well as future agents.

It would be a mistake however to alter Trove's standard method of
communication to please the new Unified Agent. In general, we should
try to keep Trove speaking to guest agents in Trove's terms alone to
prevent bloat.

If Trove's communication method is in fact superior to all others,
then perhaps we should discuss using that in the unified agent framework.

Also I'd certainly be interested in hearing about whether or not you
think the C++ agent could made generic enough for any project to use.
That would be a nice win.
If footprint is the concern, I am certain you would be better off with a C implementation. C++ is full of fail in a variety of ways and offers no useful advantage for something as small as an agent ;-)

We wrote our heat agents in python because they are not long-lived processes, hence they can bleed into the guest OS memory on demand and are only typically used during bootstrapping, vs always taking a chunk off the top as a long-lived agent process would do.

Just for comparison sake, the basic footprint of an operation like sleep(100) (eg the runtime)

python:
consumes 12.5MB of virt memory and 4.3MB of resident memory.

C:
4MB of virt memory and 328k of resident memory

C++:
12.5MB of virt memory and 784k of resident memory

C++ buys 3MB of ram, compared to all the losses of the great functionality the infrastructure team has put into place. But C is a real winner for footprint, especially since the C library (the virt part) will almost always be demanded by some application via the dynamic loader.

Regards
-steve

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to