On Mon, Nov 7, 2011 at 4:42 PM, Ferenc Kovacs <tyr...@gmail.com> wrote:

>
>
> On Mon, Nov 7, 2011 at 11:14 AM, Ferenc Kovacs <tyr...@gmail.com> wrote:
>
>>
>>
>> On Thu, Nov 3, 2011 at 8:15 PM, Ferenc Kovacs <tyr...@gmail.com> wrote:
>>
>>>
>>>
>>> On Thu, Nov 3, 2011 at 7:43 PM, Stefan Marr <p...@stefan-marr.de> wrote:
>>>
>>>> Hi Ferenc:
>>>>
>>>> On 03 Nov 2011, at 19:01, Ferenc Kovacs wrote:
>>>>
>>>> > Of course there are ways to improve the current setup, I listed those
>>>> ideas
>>>> > at https://wiki.php.net/rfc/jenkins#future_plans
>>>>
>>>> Very nice.
>>>> I don't know Jenkins, but would it be possible to mail the author of a
>>>> change in case it breaks something? I just had such a case, where my local
>>>> setup was insufficient to catch it.
>>>>
>>>> Thanks a lot
>>>> Stefan
>>>>
>>>>
>>> It is, and as I mentioned in the RFC the email notification can be
>>> enabled when needed, I didn't wanted to do that before introducing the RFC.
>>> There is a slight problem however: if we don't build each and every
>>> revision (which can happen, if two commit happens before the build starts,
>>> which can happen more easily if another build is in progress for that job,
>>> because then jenkins will just queue the build, not execute it right away
>>> for obvious reasons), so there is a slight chance for "blaming" the wrong
>>> person.
>>> The usual setup is that you have a mailing list/alias which always gets
>>> a mail about the build results (which can also be customized in detail,
>>> when to mail, etc.) and optionally you can notify the person whose commit
>>> break the build.
>>>
>>> As Klaus pointed out, there are other possibilities for notification,
>>> currently we were using the irc plugin, so if you hang around on #php.qaon 
>>> efnet, you could see the build results or ask phpcibot about the status
>>> of the builds, However I just noticed that there is no way to restrict who
>>> can start builds through the irc bot(see
>>> https://issues.jenkins-ci.org/browse/JENKINS-5931), so I turned it off.
>>>
>>> 20:13 -!- phpcibot [~PircBotX@80.82.121.132] has quit [Remote host
>>> closed the connection]
>>>
>>> :(
>>>
>>> --
>>> Ferenc Kovács
>>> @Tyr43l - http://tyrael.hu
>>>
>>
>> I reported https://issues.jenkins-ci.org/browse/JENKINS-11606 on the
>> jenkins issue tracker, and they fixed it super fast, so phpcibot is back
>> again. \o/
>>
>>
>>
>> --
>> Ferenc Kovács
>> @Tyr43l - http://tyrael.hu
>>
>
> Hi.
>
> I would like to set up a windows slave also, as I mentioned in the RFC.
> I was informed that oti1 is mostly idle, so if somebody could give me
> access there, I would set up a jenkins slave, and the neccessary
> dependencies there.
> Thanks!
>
> --
> Ferenc Kovács
> @Tyr43l - http://tyrael.hu
>

Yesterday we were talking about enabling more extensions and adding pecl
into the mix:
http://www.mail-archive.com/internals@lists.php.net/msg54220.html
I think it would be a good idea to try to fix the currently failing tests.
http://ci.qa.php.net/job/php-src-5.3-matrix-tests/35/testReport/?
http://ci.qa.php.net/job/php-src-5.4-matrix-tests/88/testReport/?
http://ci.qa.php.net/job/php-src-trunk-matrix-tests/52/testReport/?

There is only three failing tests left on the debian slaves:

For 5.3
ext/phar/tests/phar_oo_005.phpt.Phar and RecursiveDirectoryIterator 0.01 20
ext/spl/tests/bug60082.phpt.Bug #60082 (100% CPU / when using references
with ArrayObject(&$ref))

For 5.4 and trunk
ext/standard/tests/general_functions/bug39322.phpt.Bug #39322
(proc_terminate() loosing process resource)

For the freebsd failures, some of them can be tackled from the test (the
locale related tests), but I see some things that need fixing in the code,
rather than in the tests:
https://bugs.php.net/bug.php?id=60186
https://bugs.php.net/bug.php?id=60222

I think I also come up with a solution for the xfail issue (where do we
count those):
I would add support to run-tests to be able only run the XFAILS, or skip
them.
This way I could run the testsuite twice(either adding another axis to the
matrix config, or duplicating the php-src-$version-matrix-tests jobs in
two), and we could get a distinct report about the failing tests and the
xfails.
Having an xfail would mean a test success(in the junit.xml), having an
xfail test pass would be a fail((in the junit.xml)) but wouldn't fail the
build, only make it unstable.
This would mean that if we have failing tests ( not xfail tests, but tests
which are expected to pass failing instead)
our php-src-$version-matrix-tests build would result in a failure (red
ball).
Having only passing xfails, and no other failure would make our
php-src-$version-matrix-tests would result in an unstable build (yellow
ball).
Having no xfail tests present, and everything else passing would result
in successful and stable build (blue ball).

-- 
Ferenc Kovács
@Tyr43l - http://tyrael.hu

Reply via email to