Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-19 Thread Daniel Golle
Now after not thinking about it for a few days and returning to it now, I see where to get a first detail done properly: A decent remote-control library, preferibly written in Python. Something like this is probably part of every mass-production plant, but i'm not aware of a generic FOSS solutio

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-17 Thread Daniel Golle
On 01/17/2011 04:19 PM, Travis Kemen wrote: I believe that I can create dependent builds, ie if a build completes then queue the test job, which could be assigned to a single slave. (This is off the top of my head so it may vary) Sounds nice! I also believe the best is to generate test-plans fo

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-17 Thread Travis Kemen
I believe that I can create dependent builds, ie if a build completes then queue the test job, which could be assigned to a single slave. (This is off the top of my head so it may vary) Travis On Mon, Jan 17, 2011 at 3:01 AM, Daniel Golle wrote: > > > I don't believe there has to be a lot of co

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-17 Thread Daniel Golle
I don't believe there has to be a lot of communication with the build-bot Why not? I mean, it is not *much* communication bandwidth-wise, neither does it have stringent latency requirements. But building on the same machine that later orchestrates the test makes more sense than distributing t

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-16 Thread Florian Sesser
Hi Daniel! Sorry for raining on your parade... On 16/01/11 19:47, Daniel Golle wrote: > I don't believe there has to be a lot of communication with the > build-bot Why not? I mean, it is not *much* communication bandwidth-wise, neither does it have stringent latency requirements. But building on

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-16 Thread Daniel Golle
Good to see this resonating in some form :) I also like the idea of having it distributed. I don't believe there has to be a lot of communication with the build-bot, so buildbot and testbot can be two seperate animals. testbot should probably have an interface accessible by buildbot to add jobs

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-12 Thread Florian Sesser
Hi! On 10/01/11 05:22, rellis wrote: > If you google for 'linux wireless automated testbed' you'll find a plethora > of entries. It seems to be a research topic at a number of universities. Yes, but I highly doubt any of these can be talked into giving their stuff away to something that does n

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-12 Thread Florian Sesser
The plastic router cloud! Way awesome! I know building stuff is big fun, but before anyone gets her/his soldier iron out, this could maybe be tried with an emulator? Shouldn't there be a gazillion emulators out there for the obscurest architectures? I recall having seen ARM ones even in the debian

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-12 Thread Florian Sesser
Nice! Thank you Bastian! Please be aware that * I haven't tested the USB-to-serial HW that I had suggested with Linux. I do not own one of those, but I believe they work out of the box. * As you may know I am no frequent committer to OpenWRT nor lurker on the ML, you probably know better than me

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-09 Thread rellis
(I guess I should learn to proofread my outgoing mail before I work on testing :( ) ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-09 Thread rellis
On Jan 4, 2011, at 2:28 AM, Daniel Golle wrote: > i'm day-dreaming about an automated testing environment for OpenWrt > for some days. On Jan 4, 2011, at 2:28 AM, Daniel Golle wrote: > i'm day-dreaming about an automated testing environment for OpenWrt > for some days. If you google for 'l

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-09 Thread Travis Kemen
I will make some time to make the buildbot work with it. Travis On Sun, Jan 9, 2011 at 11:34 AM, Bastian Bittorf wrote: > > > but I've recently came across a usb2-to-four-times-serial for 39 > > euros [1] > > Good idea. @work we have enough space/power to make > this. So i will setup: > > 1 x Co

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-09 Thread Bastian Bittorf
> > but I've recently came across a usb2-to-four-times-serial for 39 > euros [1] Good idea. @work we have enough space/power to make this. So i will setup: 1 x Computer 4 x Router (Linksys WRT54GL, TP-Link 1043ND, DIR-300, RouterStation Pro) 4 x Serial soldered to routers 1 x Internet 1 x Ethern

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-04 Thread Travis Kemen
It is being used by some developers. Trunk is going though some libtool changes that are affecting some packages as well as I just added a new buildslave (buildslave1) that failed out a couple builds. The buildbot does report failed builds to the #openwrt-devel irc channel. Travis On Tue, Jan 4,

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-04 Thread Florian Sesser
Right, i meant something exactly like that. Doesn't look too good at the moment, though. Is the thing being used? http://buildbot.openwrt.org:8010/grid Does it already send out flames to developers? It should, IMO. I think watching over the build process - as buildbot already does - gets you ve

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-04 Thread Travis Kemen
We currently are using the buildbot software for automating the snapshot builds. http://buildbot.openwrt.org:8010/ It would be possible to script some test cases and have them run after each successful build but we currently do not have a box that this could be done with. The test lab would need ne

Re: [OpenWrt-Devel] automated testing brainstorming

2011-01-04 Thread Florian Sesser
Good idea, some continuous integration system could make all of of our lives easier. That said, it could make our lives harder, also. Isn't there lots and lots of stuff like that already around? Might Hudson [1] be the thing we want here? It hooks into the RCS and builds the software for many

[OpenWrt-Devel] automated testing brainstorming

2011-01-04 Thread Daniel Golle
i'm day-dreaming about an automated testing environment for OpenWrt for some days. automated testing with JUnit and ruby-unit has made my life much better and the time-loss when developing on OpenWrt without decent automated testing can be very frustrating. sure, everybody can just setup a bunch of