Right, i meant something exactly like that.
Doesn't look too good at the moment, though. Is the thing being used?
http://buildbot.openwrt.org:8010/grid
Does it already send out flames to developers? It should, IMO.
I think watching over the build process - as buildbot already does -
gets you very far already, at least for the snapshots. Complicated
regressions are hard to find and write tests for, anyway; a failing
compile process is detected easily and consistently.
For a 'real' testbed, of course, real devices are a must, you're right.
Maybe some sponsor can chip in and provide a bit of power and shelf
space (need not be rack, ikea is fine)? A fast internet connection is
not required.
A friend of mine is happy using a combination of this
GEMBIRD SIS-PM SilverShield (6 outlets, i think only 4 of which can be
controlled via USB, 30 euros on amazon (100 if you want LAN instead of USB)
with that
http://sispmctl.sourceforge.net/
and USB->serial connectors also aren't too expensive, I'm not into that,
but I've recently came across a usb2-to-four-times-serial for 39 euros [1]
Twice both of that plus a few cables and an ikea(TM) billy is about
250 €, and gives 8 times serial access and remote power. 100 euros per
year should be more than enough for paying the energy bills (using
renewable energy, i hope!). And the thing blinkenlights nicely all of
the tiem.
Maybe that's a nice weekend project for one of the bazillon
computer/it/dev service shops out there? Maybe one of those who also
offer their services on the internet? And would like a text link on the
OpenWRT homepage for awesome fame and glory?
Greetings, and a happy new year everyone!
Flo
[1]
https://www.pmr-funkgeraete.de/artikel/artikel-3546-kategorie-111-seite-.htm
(de)
Travis Kemen wrote:
We currently are using the buildbot software for automating the snapshot
builds. http://buildbot.openwrt.org:8010/ It would be possible to script
some test cases and have them run after each successful build but we
currently do not have a box that this could be done with. The test lab would
need networkable power so we powercycle the hardware remotely as needed. It
would also need a serial connection to all the devices.
Travis
On Tue, Jan 4, 2011 at 2:28 AM, Daniel Golle <daniel.go...@gmail.com> wrote:
i'm day-dreaming about an automated testing environment for OpenWrt
for some days. automated testing with JUnit and ruby-unit has made my
life much better and the time-loss when developing on OpenWrt without
decent automated testing can be very frustrating.
sure, everybody can just setup a bunch of scripts to test-through
whatever, but sharing the test-cases and would be great.
therefore we need a common language to specify test-cases. i want to
share the ideas i had so far. i'd be glad to hear your opinion and
anything your feel being related to this idea.
why?
- increase quality
- have better ways to find, reproduce and communicate information about
bugs
- keep us busy with developing, less time spending doing repetitive
things and waiting for results
how?
- have a test-host connected to one or more OpenWrt-based test-clients
(via serial and eventually using a reset-switch)
- let the host control and record what happens on the clients
- maybe each openwrt package can come with test-cases?
- maybe we can have a test-spec for each hardware target?
- possibility to specify multi-device-test cases, like have one be an
wifi ap and the other be wifi client
- have quantitative metrics and reports for some tests (like wifi
data-throughput)
- in case of a non-satisfying test-result, a bug-report can be
generate which, after approval, would be pushed into the bug-tracker
test-cases could be organized in an object-oriented fashion, i'm
thinking of an abstract testing-framework running on the test-host
which processes YAML or XML files holding the information of the tests
to be executed. i'd implement that in Ruby using ruby-unit.
dreams...
- have a hardware test-farm
- have a web-interface like tinderbox.mozilla.org
please share whatever comes up in you!
_______________________________________________
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
------------------------------------------------------------------------
_______________________________________________
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
_______________________________________________
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
