That's a clever idea Salvatore, I like it :)
My only real concern would be that we chose a templating language that will
be an easy dependency to fulfill on most distros, as I am guessing the CLI
to be packaged as part of the "client package", which will have to run on
any system that talks to quantum (both orchestrators and core openstack
services like nova). That definitely seems doable though.
For a while I've been meaning to redo the CLI output to be a bit more
friendly to simple shell scripting as well... so that might be a good thing
to take into consideration as long as we're redoing the output.
This will be a great improvement, as CLI unit tests are definitely a gap.
In fact, we recently ran into a CLI bug when doing a alpha deployment of
Quantum at a customer yesterday.... just a little too late :)
Dan
On Thu, Aug 11, 2011 at 8:26 AM, Salvatore Orlando <
salvatore.orla...@eu.citrix.com> wrote:
> Hi, ****
>
> ** **
>
> I’m writing the unit tests for CLI. Verifying successful completion of API
> operations implies checking the output of the CLI, which could be tedious,
> error-prone, and not easy to maintain. ****
>
> For this reason I’m thinking of introducing Cheetah templates for
> generating CLI output. This will simplify the implementation of the CLI
> itself, and terribly reduce the maintenance burden on unit tests.****
>
> ** **
>
> Has anybody strong reasons against using Cheetah? ****
>
> ** **
>
> Also, the CLI can currently operates in two modes:****
>
> **· **Direct, by directly calling the plugin****
>
> **· **API, by contacting the API server****
>
> Do we still have a use case for the first operational mode?****
>
> ** **
>
> Since this post is mainly related to implementation, let’s talk some code:
> ****
>
> ** **
>
> Current CLI operations for retrieving network list (direct to plugin and
> through API): ****
>
> ** **
>
> def list_nets(manager, *args):****
>
> tenant_id = args[0]****
>
> networks = manager.get_all_networks(tenant_id)****
>
> print "Virtual Networks on Tenant:%s\n" % tenant_id****
>
> for net in networks:****
>
> id = net["net-id"]****
>
> name = net["net-name"]****
>
> print "\tNetwork ID:%s \n\tNetwork Name:%s \n" % (id, name)****
>
> ** **
>
> ** **
>
> def api_list_nets(client, *args):****
>
> tenant_id = args[0]****
>
> res = client.list_networks()****
>
> LOG.debug(res)****
>
> print "Virtual Networks on Tenant:%s\n" % tenant_id****
>
> for n in res["networks"]:****
>
> net_id = n["id"]****
>
> print "\tNetwork ID:%s\n" % (net_id)****
>
> ** **
>
> Same code revised using cheetah template:****
>
> ** **
>
> def list_nets(manager, *args):****
>
> tenant_id = args[0]****
>
> networks = manager.get_all_networks(tenant_id)****
>
> builder=NetworkBuilder()****
>
> nw_list = [builder.build(network, net_detail=True,
> port_detail=False)['network']****
>
> for network in networks]****
>
> res = dict(networks=nw_list) ****
>
> output = prepare_output("list_nets", tenant_id, res)****
>
> print output****
>
> ** **
>
> ** **
>
> def api_list_nets(client, *args):****
>
> tenant_id = args[0]****
>
> res = client.list_networks()****
>
> output = prepare_output("list_nets", tenant_id, res)****
>
> print output****
>
> ** **
>
> The chetaah template file:****
>
> ** **
>
> ## Cheetah template for cli output****
>
> #if $cmd == 'list_nets'****
>
> Virtual Networks on Tenant $tenant_id****
>
> #for $network in $networks****
>
> \tNetwork ID: $network.id****
>
> #end for ****
>
> #end if****
>
> ** **
>
> Unit tests for the same operation:****
>
> ** **
>
> def _verify_list_networks(self):****
>
> # Verification - get raw result from db****
>
> nw_list = db.network_list(self.tenant_id)****
>
> networks=[dict(id=nw.uuid, name=nw.name) for nw in nw_list]***
> *
>
> # Fill CLI template****
>
> output = cli.prepare_output('list_nets', self.tenant_id,****
>
> dict(networks=networks))****
>
> # Verify!****
>
> # Must add newline at the end to match effect of print call***
> *
>
> self.assertEquals(self.fake_stdout.make_string(), output +
> '\n')****
>
> ****
>
> def test_list_networks(self):****
>
> try: ****
>
> cli.list_nets(self.manager, self.tenant_id)****
>
> LOG.debug("Operation completed. Verifying result")****
>
> LOG.debug(self.fake_stdout.content)****
>
> self._verify_list_networks()****
>
> except: ****
>
> LOG.exception("Exception caught: %s", sys.exc_info())****
>
> self.fail("test_list_network_api failed due to an exception")*
> ***
>
> ** **
>
> ****
>
> def test_list_networks_api(self):****
>
> try: ****
>
> cli.api_list_nets(self.client, self.tenant_id)****
>
> LOG.debug("Operation completed. Verifying result")****
>
> LOG.debug(self.fake_stdout.content)****
>
> self._verify_list_networks() ****
>
> except: ****
>
> LOG.exception("Exception caught: %s", sys.exc_info())****
>
> self.fail("test_list_network_api failed due to an exception")*
> ***
>
> ** **
>
> The code has been already pushed to lp:~netstack/quantum/quantum-unit-tests
> ****
>
> ** **
>
> By the way, many thanks to the author of the client library for adding an
> option for having a fake connection for testing! It’s a shame I realized it
> after having stubbed out part of do_request J****
>
> ** **
>
> Cheers,****
>
> Salvatore****
>
> --
> Mailing list: https://launchpad.net/~netstack
> Post to : netstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~netstack
> More help : https://help.launchpad.net/ListHelp
>
>
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dan Wendlandt
Nicira Networks, Inc.
www.nicira.com | www.openvswitch.org
Sr. Product Manager
cell: 650-906-2650
~~~~~~~~~~~~~~~~~~~~~~~~~~~
--
Mailing list: https://launchpad.net/~netstack
Post to : netstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~netstack
More help : https://help.launchpad.net/ListHelp
