IMO, the log collection topic is a CI/Releng function rather than the test framework’s responsibility, we should not discuss it only during the Functest weekly since we would lose audience. @David, will you bring this discussion to the infra or technical weekly meeting?
Thanks, Jose > On 17 Oct 2017, at 00:32, Srikanth Vavilapalli > <[email protected]> wrote: > > Hi > > Can we have a quick discussion in any weekly meeting (sdnvpn, functest?) on > this topic to pick the best approach to move forward? Also Xu Dan has raised > few other concerns on the location of log files and option for disabling the > gathering logs…etc. > > Thanks > Srikanth > > > > From: [email protected] > [mailto:[email protected]] On Behalf Of Jose Lausuch > Sent: Friday, October 13, 2017 4:28 AM > To: Tim Irnich <[email protected]> > Cc: [email protected] > Subject: Re: [opnfv-tech-discuss] [functest] [sdnvpn] Proposal for removing > installer dependent information in the test tools > > Hi Tim, > > My only concern is that by implementing this function inside the test case, > we might run into exceptions raised if there is a problem with SSH keys or > whatever other problem accessing the nodes. > > Collecting logs when the test fails makes sense, but it doesn’t harm if we > collect it when it passes as well. Anyway, if we do it only when having > failures, I imagine that logic could also be implemented post-job by checking > the "criteria": “FAIL" value on the DB and executing some j-job to collect > the logs. This logs could be uploaded to artifact repo for further analysis > as we do for the Functest logs. > > Regards, > Jose > > > > > > On 13 Oct 2017, at 09:27, Tim Irnich <[email protected] > <mailto:[email protected]>> wrote: > > Jose, > > I see the point of avoiding log collection in the code for test execution to > avoid false negatives in case something with the log collection itself goes > wrong (although I’m wondering a bit if that cannot also be handled by > carefully implementing the test pass/fail criteria). > > There are however cases where we’ll want to collect logs only if a test > fails, in order to enable a first tier of offline troubleshooting. You’ll > remember the gather_logs function Niko implemented for the bgpvpn tests – > that is one example. Can that effectively be handled if we separate things as > you suggest? > > Regards, Tim > > > From: [email protected] > <mailto:[email protected]> > [mailto:[email protected] > <mailto:[email protected]>] On Behalf Of Jose Lausuch > Sent: Friday, October 13, 2017 09:16 > To: xudan (N) <[email protected] <mailto:[email protected]>>; Morgan, Jack > <[email protected] <mailto:[email protected]>> > Cc: [email protected] > <mailto:[email protected]> > Subject: Re: [opnfv-tech-discuss] [functest] [sdnvpn] Proposal for removing > installer dependent information in the test tools > > Hi, > > When it comes to log collection, I strongly believe it should be done > post-job in our CI pipeline, not as part of the test case. Users can always > collect logs manually regardless of the installer tools they use… > > And regarding making it automated in OPNFV after Functest/Yardstick > execution, would it make sense to re-use the PDF (Pod Descriptor File)? > Otherwise, we are duplicating config files like pod.yaml or new files with > information about the servers… > @Jack: is there a possibility to include login information in the PDF (user, > password, path to private key, …) of the nodes already deployed? > > Regards, > Jose > > > > > > On 12 Oct 2017, at 03:50, xudan (N) <[email protected] > <mailto:[email protected]>> wrote: > > Hi Srikanth, > > I have one question. Are the paths of all these log files constant for > different environment (Apex, Fuel and commercial deployments)? > If all paths for different deployments are the same, then using config file > to login and getting files can work. > If not, there will be some errors even though it can login with the config > file. > > One more question, what logs do SDNVPN get from all nodes? Are they useful > for users? If not, can we have an option to disable it? > > Thanks > Dan Xu > > From: [email protected] > <mailto:[email protected]> > [mailto:[email protected] > <mailto:[email protected]>] On Behalf Of limingjiang > Sent: Thursday, October 12, 2017 9:24 AM > To: Srikanth Vavilapalli > Cc: [email protected] > <mailto:[email protected]> > Subject: [opnfv-tech-discuss] 答复: [functest] [sdnvpn] Proposal for removing > installer dependent information in the test tools > > Hi Srikanth, > > Yardstick can use a global pod.yaml for test cases. > Since each test case default use the “pod.yaml” located in > “/etc/yardstick/pod.yaml”. so if you put “pod.yaml” here, it can apply to > each test case. > The picture you show below is how yardstick test suite customize the input > parameters, so it also support each test case with different “pod.yaml” > if you give each test case different “pod.yaml” > > BR, > Rex > > +-------------------------------------------------------------------------------------------+ > <image001.png> > + Mingjiang Li (Rex) Mobile: +86 13761275017 > + Shanghai Institute, Huawei > + No. 2222, Xinjinqiao Road, Pudong, Shanghai, 201206, P.R.China > +-------------------------------------------------------------------------------------------+ > > 发件人: [email protected] > <mailto:[email protected]> > [mailto:[email protected] > <mailto:[email protected]>] 代表 Srikanth Vavilapalli > 发送时间: 2017年10月12日 9:03 > 收件人: Jose Lausuch; Georg Kunz > 抄送: [email protected] > <mailto:[email protected]> > 主题: Re: [opnfv-tech-discuss] [functest] [sdnvpn] Proposal for removing > installer dependent information in the test tools > > Thanks everyone for your inputs. > > So if Yardstick based approach is the preferred one, then I am thinking of > extending the existing Deployment Factory class with a new generic INSTALLER > type (something like “config-file” or so) which will provide the same > interface as other adapters (ApexAdapter, FuelAdapter…etc), but instead reads > from the a configured pod.yaml file to provide Node information. Plz let me > know if you see any issues with this approach. > > One quick question on Yardstick: Looks like Yardstick accepts the custom > pod.yaml file on a per test case basis as shown in the below example. Can it > also accept a global pod.yaml file that can be applied to all or a group of > test cases. > - > > file_name: opnfv_yardstick_tc043.yaml > > constraint: > > installer: xxx > > pod: xxx-pod1 > > task_args: > > xxx-pod1: '{"pod_info": "etc/yardstick/.../pod.yaml", > > "host": "node1.LF","target": "node2.LF"}' > > > Thanks > Srikanth > > From: [email protected] > <mailto:[email protected]> > [mailto:[email protected] > <mailto:[email protected]>] On Behalf Of Jose Lausuch > Sent: Wednesday, October 11, 2017 10:45 AM > To: Georg Kunz <[email protected] <mailto:[email protected]>> > Cc: [email protected] > <mailto:[email protected]> > Subject: Re: [opnfv-tech-discuss] [functest] [sdnvpn] Proposal for removing > installer dependent information in the test tools > > Hi, > > I would vote for having something similar to Yardstick [1] but centralized in > Releng with an easy python lib that enables functions like SCP things to/from > the deployed nodes. > > For your third point, log collection shouldn’t be done at test case level. It > should be performed by CI after running the test tools, otherwise you can a > false negative when running those test on non-OPNFV installers. > > Regards, > Jose > > [1] > https://raw.githubusercontent.com/opnfv/yardstick/master/etc/yardstick/nodes/fuel_virtual/pod.yaml > > <https://raw.githubusercontent.com/opnfv/yardstick/master/etc/yardstick/nodes/fuel_virtual/pod.yaml> > > > On 11 Oct 2017, at 18:08, Georg Kunz <[email protected] > <mailto:[email protected]>> wrote: > > Hi, > > Just to highlight this, from a Dovetail/CVP perspective, the important aspect > is that there are no dependencies on OPNFV-specific resources/lib in order to > be able to run test cases against commercial non-OPNFV deployments. > > Having to write an adapter for a particular commercial deployment before you > can run Dovetail is obviously not really an option. So, for tests which > require SSH/SCP access, we need to think about... > If the adapter can be parameterized, so that we can make it a configuration > option, e.g., specifying login credentials, source and target directories, > etc., similarly to Yardstick. > Reuse what Yardstick is using? > If the test case be parameterized such that it does not attempt to gather > logs if used for certification? (limited use, of course) > … > > Cheers > Georg > > From: [email protected] > <mailto:[email protected]> > [mailto:[email protected] > <mailto:[email protected]>] On Behalf Of Jose Lausuch > Sent: Wednesday, October 11, 2017 4:40 PM > To: Beierl, Mark <[email protected] <mailto:[email protected]>> > Cc: [email protected] > <mailto:[email protected]> > Subject: Re: [opnfv-tech-discuss] [functest] [sdnvpn] Proposal for removing > installer dependent information in the test tools > > Hi, > > With regards to Functest, you can run it on any OpenStack deployment as long > as you provide a proper RC file and meet the requirements on the jumphost > (docker, connectivity to the deployment, …). > > However, in some cases, some test cases from feature projects require SSH > access to the deployment and to make things centralized, the deployment > handler was created [1]. This is a library that allows users to get the > number of nodes from the deployment, functions to SCP things from the nodes > and some other utils. The bad part of it is that it only supports Apex, Fuel > and OSA for now… unless someones volunteers to write the other adapters for > joid, mcp, compass osa.. This library might be used to extract the desired > logs after Functest/Yardstick runs in CI to place them in artifact repo and > post-analize. > > Regards, > Jose > > [1] https://git.opnfv.org/releng/tree/modules/opnfv/deployment > <https://git.opnfv.org/releng/tree/modules/opnfv/deployment> > > > > > On 11 Oct 2017, at 16:23, Beierl, Mark <[email protected] > <mailto:[email protected]>> wrote: > > Hello, > > StorPerf very much relies on knowledge of the installer to gather information > about the block storage underlay. For example, the number of Ceph nodes, or > even Ceph vs. LVM, is very relevant to the final report. I also wish there > were an installer agnostic method of collecting this information as right now > I keep that code in the ci/daily.sh and other scripts. > > With the new releng repository being created, perhaps it is time to start > moving some of the installer specific code there? I also see that being of > benefit when adding XCI support, as technically that would be yet another > type of installer. > > Regards, > Mark > > Mark Beierl > SW System Sr Principal Engineer > Dell EMC | Office of the CTO > mobile +1 613 314 8106 <tel:1-613-314-8106> > [email protected] <mailto:[email protected]> > > On Oct 11, 2017, at 02:25, xudan (N) <[email protected] > <mailto:[email protected]>> wrote: > > Hi Srikanth, > > As I know, some Yardstick test cases also need to login nodes. Yardstick uses > a file providing all the login information. > You can refer to > https://github.com/opnfv/yardstick/tree/master/etc/yardstick/nodes > <https://github.com/opnfv/yardstick/tree/master/etc/yardstick/nodes> which > gives some examples. > Hope this will help you. > > BR > Dan Xu > > From: Srikanth Vavilapalli [mailto:[email protected] > <mailto:[email protected]>] > Sent: Wednesday, October 11, 2017 12:28 PM > To: [email protected] > <mailto:[email protected]> > Cc: Tim Irnich; xudan (N) > Subject: [functest] [sdnvpn] Proposal for removing installer dependent > information in the test tools > > Hi > > I am looking into Jira ticket “SDNVPN-181 > <https://jira.opnfv.org/browse/SDNVPN-181>: Function "gather_logs" restricts > to Apex and Fuel”, which raises concerns on having installer dependent logic > in the sdnvpn repo. The issue is, at the end of the sdnvpn test execution, we > are invoking gather_logs() utility which internally tries to gather the > information about all the OpenStack nodes based on the configured > INSTALLER_TYPE in order to run the fetch_logs.sh script on the target > OpenStack nodes > (https://gerrit.opnfv.org/gerrit/gitweb?p=sdnvpn.git;a=blob;f=sdnvpn/lib/utils.py;h=ad0714ea9dd40ee8305cd17e42695f0176e88328;hb=HEAD#l215 > > <https://gerrit.opnfv.org/gerrit/gitweb?p=sdnvpn.git;a=blob;f=sdnvpn/lib/utils.py;h=ad0714ea9dd40ee8305cd17e42695f0176e88328;hb=HEAD#l215>) > > So, the jira ticket proposes to accept all the needed information about the > OpenStack controllers, compute nodes and the associated username, keys…etc. > in a file format such that these tests can also be run on OPNFV based > commercial products deployed with their custom deployment tools. > > So in general, in the test tools, is there any need to have awareness of what > installers being used when we all care about the target OpenStack node IPs, > associated attributes and jumphost IP (in some cases)? > > I would like to get the community opinion here. Appreciate your inputs. > > Thanks > Srikanth > > _______________________________________________ > opnfv-tech-discuss mailing list > [email protected] <mailto:[email protected]> > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > <https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss> > > _______________________________________________ > opnfv-tech-discuss mailing list > [email protected] <mailto:[email protected]> > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > <https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss> > > _______________________________________________ > opnfv-tech-discuss mailing list > [email protected] <mailto:[email protected]> > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > <https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss>
_______________________________________________ opnfv-tech-discuss mailing list [email protected] https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
