2011/2/16 Zygmunt Krynicki <zygmunt.kryni...@linaro.org>: > W dniu 15.02.2011 21:01, Jim Huang pisze: > > Hi Jim, great work!
hi Zygmunt, Thanks. It is my pleasure to work with Linaro validation team. >> ** Why can't we execute LAVA/Abrek directly on Android devices? > I agree that direct abrek is not the right solution for Android. I think > there is a class of use cases that also falls into this category: > * testing early silicon > * testing very primitive systems > * testing other foreign systems > They all have either no python or running python is not desirable. Exactly. Thanks for your explanation. [...] > Currently LAVA is just taking shape. Things like server and client are still > not very well defined. You don't have to be constrained by this. The only > thing that is somewhat well defined is abrek that was produced in the > previous cycle. Abrek was designed to run _on_ the device. For use cases > where Abrek is just interacting with the test device we may want to extend > it sensibly to clearly separate those cases where necessary while retaining > as much common code and user interface as possible. > > There are a few things to consider here: > > Device context: Currently abrek has some logic to probe the running system > for context information. Context is loosely defined as the collection of > relevant software and hardware information that might be interesting to > analyze or that can be used to connect distinct test results in post-process > analysis. If we are just interacting with a remote device we need to > determine this information in some other way _and_ ensure we're not adding > any dummy information from the "host" system. > > Remote device registration: A single host may talk to possibly large number > of such devices. At the very least we should plan how we intend to manage > the process of defining/adding/removing a device and how to allow remote > tests to be invoked on registered devices, if possible (if the device > supports this test). > > Test classes: So far abrek tests could run on any Linux box (where any was > poorly defined as "any ubuntu-like system". This is no longer the case so we > should be able to classify tests (and devices) somehow. > It sounds reasonable. Do you think whether the above changes are progressive to current LAVA implementation or not? In Android, we can even re-use the infrastructure of ADB and DDMS [1] to provide remote device registration service. > IMHO we could define a new tool (or just a new sub-command class for abrek), > say abrek-remote that will be used instead of the normal abrek call. > Yes, I like the idea. > Another crazy option would be to expose LAVA Job Dispatcher directly and > allow people to run jobs. In this case one job would use abrek and some > other tools to invoke tests, process results and send them to the dashboard > while other job (one for android) would not run abrek at all, instead it > would call some other helper, while still reusing identical components for > "process results and send to result storage" phases. This is still in flux [...][ > Another upside is that such job definitions (and any required LAVA plugins) > would just integrate with the rest of LAVA (farm backend, frontend, etc). > You could work on a job description on your workstation and once it's > finished you could add it to a job scheduler for automatic processing. > Agree. >> ** Show me the use case >> The attached patch is just trivial proof-of concept implementation >> done by Jeremy Chang that adds a 'monkey' >> test definition file for abrek. Once TCP/IP or USB is ready, for >> Android's monkey testing, the procedure is like >> as following: >> (1) abrek run monkey >> (2) abrek dashboard put /anonymous/ monkey1297752359.0 > > Could you please forgive my android ignorance and tell me how I can run this > test? Please include any hardware/software I should have. Can I run this on > a beagle board with some android installation? Do I need a real android > phone? This is just so that I can participate in the discussion and not make > clueless and pointless arguments later. Jeremy, could you prepare the testing environment using Android emulator? We could write down the instructions in wiki for reference. >> Any suggestion is appreciated. Thank you in advance. > > I hope we can work on making this solid. Sure! Cheers, Jim Huang (jserv) http://0xlab.org/ [1] DDMS: http://developer.android.com/guide/developing/tools/ddms.html _______________________________________________ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
