Hello list,
Regarding integration of LAVA [1] architecture with Android devices,
we would like to reuse the existing
infrastructure and framework design. Abrek [2] is a great testsuite
framework for running test cases and
benchmarks. However, due to the restrictions of unusual Android
runtime, we consider to introduce the
agent-based remote validation invocation mechanism for LAVA as the
extension. Also, the proof-of-concept
implementation is attached.
** Why can't we execute LAVA/Abrek directly on Android devices?
LAVA/Abrek is written in Python, which implies there must be a solid
Python runtime for Android. CPython is
verified and well-designed, but it is not well tested on Android. In
fact, Android has its own libc implementation,
bionic, which is the minimal and special libc originally taken from
NetBSD libc. However, bionic libc only supports
limited set of POSIX C APIs, and it is almost not feasible to maintain
Linaro bionic modifications in early stage
just to satisfy CPython. The bionic libc is always changed fast by
Google engineers, and we have no idea about
their plans.
Therefore, we prefer the way not to modify Android runtime. That is,
don't execute LAVA/Abrek directly on
Android environment.
** Yet another agent?
In fact, Android already provide an elegant approach for accessing
target environment, adb (Android Debug
Bridge)[3], which is a versatile tool allowing users to manage the
state of Android-powered device. adb itself
is a client-server program that includes three components:
(1) A client, which runs on your development machine. You can invoke a
client from a shell by issuing an adb command.
(2) A server, which runs as a background process on your development
machine. The server manages communication between the client and the
adb daemon running on the device.
(3) A daemon, which runs as a background process on each device instance.
The adb protocol can be established through USB (device needs to
enable Android USB gadget driver) or
TCP/IP. adb is very solid and powerful. For example, if you would
like to test Android UI, you can use the
command on Host side:
$ adb shell monkey -v -p your.package.name 500
The above could be illustrated:
Host commands --> adb protocol (USB or TCP/IP) --> Target receives
the command --> execute "monkey"
Monkey is a program on Android device that runs on your emulator or
device and generates pseudo-random
streams of user events such as clicks, touches, or gestures, as well
as a number of system-level events.
In this example, your Android application is launched, and 500
pseudo-random events are sent to it. We call
it as "agent-based remote invocation", and it is built-in.
** So, what's agent-based remote validation invocation for LAVA?
There are three key items:
(a) Agent
(b) Remote validation invocation
(c) LAVA
We keep in mind that we make no technical impact to LAVA architecture,
and the "Agent" is just the "helper".
Originally, the client-server communication looks like the following:
LAVA server <----> LAVA client --> Abrek test suite
Since Python runtime is hard to support, the proposed model would be:
LAVA server <--> LAVA client --> Abrek test suite || adb
extension (host) <--> adb (target) -> execute command
For integrating test items and benchmarks for Android, this proposal
is running abrek on the host side.
By using Android's standard tool, adb to communicate with the adbd
(adb daemon) on target, the test case
commands can be issued and the output can return back. Besides, the
files can be pushed and pull to and
from the target by adb as well. Sometimes the results may be stored
in certain file on target, and we definitely
could couple with the case. Thus, running abrek on host side along
with "Agent" should be a workable approach.
** Show me the use case
The attached patch is just trivial proof-of concept implementation
done by Jeremy Chang that adds a 'monkey'
test definition file for abrek. Once TCP/IP or USB is ready, for
Android's monkey testing, the procedure is like
as following:
(1) abrek run monkey
(2) abrek dashboard put /anonymous/ monkey1297752359.0
In addition, if we wish to execute certain native application on
Android device, abrek could ask adb to send the
executable files and related data to target first. Then, execute it
as expected. The command looks like:
$ adb push <my-executable-file> /system/bin
$ adb push <my-data-file> /data
$ adb shell /system/bin/<my-executable-file>
The above instructions could be refined into "adb extension" as the
part of LAVA client framework.
That's all. It could be straightforward and transparent.
Any suggestion is appreciated. Thank you in advance.
Sincerely,
Jim Huang (jserv)
http://0xlab.org/
[1] https://wiki.linaro.org/Platform/Validation/LAVA/Architecture
[2] https://wiki.linaro.org/Platform/Validation/AbrekTestsuites
[3] http://developer.android.com/guide/developing/tools/adb.html
# Bazaar merge directive format 2 (Bazaar 0.90)
# revision_id: jeremy.ch...@linaro.org-20110215171535-bzd9sb3wont0il67
# target_branch: http://bazaar.launchpad.net/~pwlars/abrek/trunk/
# testament_sha1: 3effa4a7c980a9b31968fc80514a0eabc5568e0b
# timestamp: 2011-02-16 02:02:38 +0800
# base_revision_id: paul.lar...@canonical.com-20110112145515-\
# z05a1jdktss8rbtu
#
# Begin patch
=== added file 'abrek/test_definitions/monkey.py'
--- abrek/test_definitions/monkey.py 1970-01-01 00:00:00 +0000
+++ abrek/test_definitions/monkey.py 2011-02-15 17:15:35 +0000
@@ -0,0 +1,27 @@
+import re
+import abrek.testdef
+
+VERSION="20110215"
+RUNSTEPS = ['adb shell monkey -s 1 --pct-syskeys 100 --throttle 500 100']
+
+class MonkeyParser(abrek.testdef.AbrekTestParser):
+ def parse(self):
+ filename = "testoutput.log"
+ PAT = "^## Network stats: elapsed time?\W+(?P<measurement>\d+)ms"
+ pat = re.compile(PAT)
+ with open(filename) as fd:
+ for line in fd:
+ match = pat.search(line)
+ if match:
+ d = match.groupdict()
+ d['units'] = "mseconds"
+ d['result'] = "pass"
+ d['test_case_id'] = "monkey"
+ self.results['test_results'].append(d)
+
+monkeyinst = abrek.testdef.AbrekTestInstaller(deps=["monkey"])
+monkeyrun = abrek.testdef.AbrekTestRunner(RUNSTEPS)
+monkeyparser = MonkeyParser()
+testobj = abrek.testdef.AbrekTest(testname="monkey", version=VERSION,
+ installer=monkeyinst, runner=monkeyrun,
+ parser=monkeyparser)
# Begin bundle
IyBCYXphYXIgcmV2aXNpb24gYnVuZGxlIHY0CiMKQlpoOTFBWSZTWW3hhegAAej/gERUQAFY7//3
8iPfj7////BQBO92YDl0cQoDpQ0QJpGTaoek2U2Cn6mkaADQGQAwgA0Qmp6ZMjRoUHoQAAAAAAAA
1U9pGk0xMQA0AAAGgAAaYgBw00yMRhNMBDAJphGCYmQ0yNDQCSIRpqYhqMCMmpNtNTEno9TUzUab
UaDQ9TT0kAKLNYMvvSJqqDbZsmaEGYQEHgDA5VS0gDcStIz3bFxytYWtBJfJkkEUl0oobvO/5H8/
M8vKubu7cPVVy+HMi87Hl2O7x0SWr1I6ZoiUFBU9Uy0dJ+eRyEZha1Ao4ZtB00JqEoYtUUV6NgvX
G2MJj31pOu3UpxJRakjP4MIBnRpbJHfNpBGDLfQGHLdnXAtj1ZZX4kqgpj0XPSSOlw7hvS4y3BWC
rHIwsWLT4WzjCpOcz8gaKkt60UdT9iUlXDei26Vn3XSZul3e9tLcOj1oUA1Mggx9lYZwgny2EShG
WUXmaLy5gc53K61jZBH9dBd3VpLlxRlNWVdlVcIKyIf7PLDKmAziczML2Xb5DrFx44QVMwcrAZu/
jRYuVBIuqyZ8cojX14UsYKYHPk06TmyFQpsV1mYwFGGqpTIYVxCFuS2OSyEBhsl1g1l8ZXFQnTZW
IKAExhZDFgTUG5TSn6LKwa+M4tYXmOVgpnUWQtTk6t1CjEoFYFslOEqUknFkQGEISY9ya8JYCzFF
lB0mMou1igLDmjO2WcEh+ArgDpuMFxAxqVXtei9lUVRmNa7rFBaBNNqiwJ2N3n02oXNh2YtnJEYf
DOvT42d0612pXHoDqZQ0YGMbsWYQh0MKB8Qgnw9ykLh0KkI6R2xCHBAGRlC+R408njtNrZfn/QdD
NkNe1nYGn2uQHDtXtPdctDbvC6w4FvUFF8L1TGDyhjSqW/j3e+7GimJJzcMzVrFR4Qr/OAwSk84r
8NZlFccTup7fjpwRvBs68Bek7zo7xGKC1l9I71DOPyeOtAgsInIkSWydSqNcKhQazNOPi28wirz0
iGEqgwsATt8AJIYTiloqxzaTApvwcWGoT4S8i2g7FDXkyGzhRSlS1eFrRPkSuoKpw03SUJ5RJDGA
8bDlwm2Nqbcm7Tg3uklmtEdmXe2XBz0XnJMDnyjPyt5plfWXtWxDNojGWxJ502YrPFmTJ2AtWT4O
EFmEEsJtewNy+vAPV1mc3HQUPrtl8eF+uPlyOcCPDOJ0uCHNp3cLGRuBknshthkTF/nS1aXd7+3i
yaf1kdYtXSwSVxmNyTwDCXs8Qz8oqmbMQigHBkAfomEjXL38I5S+B0A3DvD7PQh2RegLtwbVj3ri
qwqXWKWKKOpYvRrrU2mwxWeCXNtujsdX3O82MOpMb6D1BbuxZ4Fmyllzag1NDxZma2hXQ+ldqULW
lCDnvYUDPdUlQi4eFTjbBySUF2QpBAMRTwX7tOAmQ7Jl15krW0NsRfUHOmh78a0oPMYUuArLGtLQ
2JSgG5RWSoeyepmTdp26+MZLVqmYrWepiGqVemF2VP3EaEHQfAZQD18FK454iLa0pI/Tstv6aFqV
c64Iglj7I5fZlqSmymO2kogrU6OMYjveqAyoJEDCOYE/G2TWbfWsFdUYtsiUxprqGDSV46xvg4zJ
MmsZKgMGDytaDTdtfTAsecNdchY3eKGb7DBa0UyIPdE5tzaEZueMjHO2GWGvCPILfSCqI4oU5pzq
LjKZ2QpXUE0tN4Dhb5IaoYIgVdAHYdP5CviFMbqjDENyUE7ILdCoZcBchV/4u5IpwoSDbwwvQA==
_______________________________________________
linaro-dev mailing list
linaro-dev@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-dev
