[PATCH] a new cgi script

[Erwan Prioul]

Hi,

I'd like to add a new cgi script into udd.
The goal of this script is to display FTBFS packages on a given architecture.
For every package, it gives the name of the package, its version, the time 
since the latest build and links to the bts, the tracker and the logs of the 
latest build. It also shows the related bugs (bug id + bug title) if any.

Let me know if I'm doing wrong.

Thanks in advance,
Erwan.

From 9cf994e38331f895fae15772225925e25d304391 Mon Sep 17 00:00:00 2001
From: Erwan Prioul 
Date: Wed, 15 Feb 2017 15:43:32 +0100
Subject: [PATCH] add first version of ftbfs.cgi

script that displays the FTBFS packages on a given architecture, with related bugs if any.
---
 web/cgi-bin/ftbfs.cgi | 247 ++
 1 file changed, 247 insertions(+)
 create mode 100755 web/cgi-bin/ftbfs.cgi

diff --git a/web/cgi-bin/ftbfs.cgi b/web/cgi-bin/ftbfs.cgi
new file mode 100755
index 000..b340e59
--- /dev/null
+++ b/web/cgi-bin/ftbfs.cgi
@@ -0,0 +1,247 @@
+#!/usr/bin/env python
+# Display FTBFS packages on given arch
+# Copyright (C) 2017, Erwan Prioul 
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU General Public License as published by the Free Software
+# Foundation, either version 3 of the License, or (at your option) any later
+# version.
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more
+# details.
+# You should have received a copy of the GNU General Public License along with
+# this program.  If not, see .
+
+import datetime
+import psycopg2
+import cgi
+import cgitb
+
+DATABASE = {
+'database': 'udd',
+'port': 5452,
+'host': 'localhost',
+'user': 'guest'
+}
+
+class AttrDict(dict):
+def __init__(self, **kwargs):
+for key, value in kwargs.iteritems():
+self[key] = value
+
+def __getattr__(self, name):
+try:
+return self[name]
+except KeyError, e:
+raise AttributeError(e)
+
+def query(query, cols, *parameters):
+try:
+conn = psycopg2.connect(**DATABASE)
+cursor = conn.cursor()
+cursor.execute(query, parameters)
+except:
+exit(1)
+for row in cursor.fetchall():
+yield AttrDict(**dict(zip(cols, row)))
+cursor.close()
+conn.close()
+
+def pretty_time_delta(when):
+seconds = (datetime.datetime.now() - when).total_seconds()
+days, seconds = divmod(seconds, 86400)
+hours, seconds = divmod(seconds, 3600)
+minutes, seconds = divmod(seconds, 60)
+if days > 0:
+return '%dd' % (days)
+elif hours > 0:
+return '%dh' % (hours)
+elif minutes > 0:
+return '%dm' % (minutes)
+else:
+return '%ds' % (seconds)
+
+def packageLine(packages, package, pending = False):
+u = package.replace('+', '%2B')
+bugs = " "
+what = 'nopatch'
+if pending:
+what = 'patch'
+if len(packages[package][what]) > 0:
+bugs = ''.join('http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=%d";>#%d - %s' % (x[0], x[0], x[1]) for x in sorted(packages[package][what]))
+return \
+'%s_%s' % (package, packages[package]['version']) + \
+'' + \
+'https://buildd.debian.org/status/package.php?p=%s&suite=sid"; title="Debian buildd status">%s ' % (u, pretty_time_delta(packages[package]['state_change'])) + \
+'[http://bugs.debian.org/cgi-bin/pkgreport.cgi?src=%s"; title="Debian bugs in source package">B]' % (u) + \
+'[http://buildd.debian.org/status/logs.php?pkg=%s"; title="Debian build logs">L]' % (u) + \
+'[https://tracker.debian.org/pkg/%s"; title="Debian Package Tracker">T]' % (u) + \
+'%s' % bugs
+
+def generatePending(packages, nb, arch):
+return \
+'%d FTBFS packages on %s with a patch pending' % (nb, arch) + \
+''.join(packageLine(packages, x, True) for x in sorted(packages.keys()) if len(packages[x]['patch']) > 0) + \
+''
+
+def generateFailing(packages, failing, nb, arch):
+html = '%d FTBFS packages on %s without patch' % (nb, arch)
+for c in sorted(failing.keys()):
+if c > 2:
+	txt = "and %d other architectures" % (c - 1)
+elif c == 2:
+txt = "and 1 other architecture"
+else:
+txt = ""
+html += \
+'%d packages are failing on %s %s' % (len(failing[c]), arch, txt) + \
+''.join(packageLine(packages, x) for x in sorted(failing[c])) + \
+''
+return html
+
+def getPackages(arch):
+# Get all FTBFS packages on given architecture
+q = """
+select distinct source, version, state_change
+from wannabuild
+where architecture = '%s' and
+  distribution = 'sid' and
+  state in ('Failed', 'Build-Attempted') and
+  

Bug#855220: [PATCH] a new cgi script

[Mattia Rizzolo]

Package: qa.debian.org
Severity: wishlist
User: qa.debian@packages.debian.org
Usertags: udd
Control: submitter -1 Erwan Prioul 
X-Debbugs-Cc: Erwan Prioul 


Re-submitting the request to the BTS, so it won't get lost until
somebody will came around at reading the proposed script and push it.
(also re-attached the attachment).  Original text follows:

On Wed, Feb 15, 2017 at 04:07:56PM +0100, Erwan Prioul wrote:
> I'd like to add a new cgi script into udd.
> The goal of this script is to display FTBFS packages on a given
> architecture.
> For every package, it gives the name of the package, its version, the
> time since the latest build and links to the bts, the tracker and the
> logs of the latest build. It also shows the related bugs (bug id +
> bug title) if any.
> 
> Let me know if I'm doing wrong.

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature

Processed: Re: [PATCH] a new cgi script

[Debian Bug Tracking System]

Processing control commands:

> submitter -1 Erwan Prioul 
Bug #855220 [qa.debian.org] [PATCH] a new cgi script
Changed Bug submitter to 'Erwan Prioul ' from 'Mattia 
Rizzolo '.

-- 
855220: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=855220
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#855220: [PATCH] a new cgi script

[Mattia Rizzolo]

On Wed, Feb 15, 2017 at 04:53:14PM +0100, Mattia Rizzolo wrote:
> (also re-attached the attachment).

... here.

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-
From 9cf994e38331f895fae15772225925e25d304391 Mon Sep 17 00:00:00 2001
From: Erwan Prioul 
Date: Wed, 15 Feb 2017 15:43:32 +0100
Subject: [PATCH] add first version of ftbfs.cgi

script that displays the FTBFS packages on a given architecture, with related bugs if any.
---
 web/cgi-bin/ftbfs.cgi | 247 ++
 1 file changed, 247 insertions(+)
 create mode 100755 web/cgi-bin/ftbfs.cgi

diff --git a/web/cgi-bin/ftbfs.cgi b/web/cgi-bin/ftbfs.cgi
new file mode 100755
index 000..b340e59
--- /dev/null
+++ b/web/cgi-bin/ftbfs.cgi
@@ -0,0 +1,247 @@
+#!/usr/bin/env python
+# Display FTBFS packages on given arch
+# Copyright (C) 2017, Erwan Prioul 
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU General Public License as published by the Free Software
+# Foundation, either version 3 of the License, or (at your option) any later
+# version.
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more
+# details.
+# You should have received a copy of the GNU General Public License along with
+# this program.  If not, see .
+
+import datetime
+import psycopg2
+import cgi
+import cgitb
+
+DATABASE = {
+'database': 'udd',
+'port': 5452,
+'host': 'localhost',
+'user': 'guest'
+}
+
+class AttrDict(dict):
+def __init__(self, **kwargs):
+for key, value in kwargs.iteritems():
+self[key] = value
+
+def __getattr__(self, name):
+try:
+return self[name]
+except KeyError, e:
+raise AttributeError(e)
+
+def query(query, cols, *parameters):
+try:
+conn = psycopg2.connect(**DATABASE)
+cursor = conn.cursor()
+cursor.execute(query, parameters)
+except:
+exit(1)
+for row in cursor.fetchall():
+yield AttrDict(**dict(zip(cols, row)))
+cursor.close()
+conn.close()
+
+def pretty_time_delta(when):
+seconds = (datetime.datetime.now() - when).total_seconds()
+days, seconds = divmod(seconds, 86400)
+hours, seconds = divmod(seconds, 3600)
+minutes, seconds = divmod(seconds, 60)
+if days > 0:
+return '%dd' % (days)
+elif hours > 0:
+return '%dh' % (hours)
+elif minutes > 0:
+return '%dm' % (minutes)
+else:
+return '%ds' % (seconds)
+
+def packageLine(packages, package, pending = False):
+u = package.replace('+', '%2B')
+bugs = " "
+what = 'nopatch'
+if pending:
+what = 'patch'
+if len(packages[package][what]) > 0:
+bugs = ''.join('http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=%d";>#%d - %s' % (x[0], x[0], x[1]) for x in sorted(packages[package][what]))
+return \
+'%s_%s' % (package, packages[package]['version']) + \
+'' + \
+'https://buildd.debian.org/status/package.php?p=%s&suite=sid"; title="Debian buildd status">%s ' % (u, pretty_time_delta(packages[package]['state_change'])) + \
+'[http://bugs.debian.org/cgi-bin/pkgreport.cgi?src=%s"; title="Debian bugs in source package">B]' % (u) + \
+'[http://buildd.debian.org/status/logs.php?pkg=%s"; title="Debian build logs">L]' % (u) + \
+'[https://tracker.debian.org/pkg/%s"; title="Debian Package Tracker">T]' % (u) + \
+'%s' % bugs
+
+def generatePending(packages, nb, arch):
+return \
+'%d FTBFS packages on %s with a patch pending' % (nb, arch) + \
+''.join(packageLine(packages, x, True) for x in sorted(packages.keys()) if len(packages[x]['patch']) > 0) + \
+''
+
+def generateFailing(packages, failing, nb, arch):
+html = '%d FTBFS packages on %s without patch' % (nb, arch)
+for c in sorted(failing.keys()):
+if c > 2:
+	txt = "and %d other architectures" % (c - 1)
+elif c == 2:
+txt = "and 1 other architecture"
+else:
+txt = ""
+html += \
+'%d packages are failing on %s %s' % (len(failing[c]), arch, txt) + \
+''.join(packageLine(packages, x) for x in sorted(failing[c])) + \
+''
+return html
+
+def getPackages(arch):
+# Get all FTBFS packages on given architecture
+q = """
+select distinct source, version, state_change
+from wannabuild
+where architecture = '%s' and
+  distribution = 'sid' and
+  state in ('F

Processed: tagging 855220

[Debian Bug Tracking System]

Processing commands for cont...@bugs.debian.org:

> tags 855220 + patch
Bug #855220 [qa.debian.org] [PATCH] a new cgi script
Added tag(s) patch.
> thanks
Stopping processing here.

Please contact me if you need assistance.
-- 
855220: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=855220
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Re: Bug#842829: UDD/uscan: devscripts needs to be upgraded to avoid some failures

[Vagrant Cascadian]

On 2016-11-01, James McCoy wrote:
> On Nov 1, 2016 11:33, "Lucas Nussbaum"  wrote:
>> I tried with the first package listed.  It works fine with devscripts
>> 2.16.8 (in testing), but not with 2.16.4~bpo8+1 (in stable-bpo, and
>> installed on ullmann.debian.org).
>>
>> To fix that, an update of that backport is needed.
>
> I've yet to do that because it would require a backport of the licensecheck
> package, too, since that was split out of devscripts.

I don't know how many packages this is impacting, but the watch file for
epoptes, which works with stretch's devscripts (2.17.1), is also failing
to find a new version on the tracker.debian.org and udd.debian.org
pages. It's a little distracting...


Thanks for all your work on QA infrastructure, this is a relatively
small annoyance in an overall tremendously useful system!


live well,
  vagrant


signature.asc
Description: PGP signature

Re: [PATCH] a new cgi script

[Paul Wise]

On Wed, Feb 15, 2017 at 11:07 PM, Erwan Prioul wrote:

> I'd like to add a new cgi script into udd.
> The goal of this script is to display FTBFS packages on a given architecture.

That sounds a bit like the buildd pages:

https://buildd.debian.org/status/architecture.php?a=amd64

> For every package, it gives the name of the package, its version, the time 
> since the latest build and links to the bts, the tracker and the logs of the 
> latest build. It also shows the related bugs (bug id + bug title) if any.

That is a fair bit more detailed than the buildd page.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise