Package: qa.debian.org
Severity: minor
Tags: patch
On Tue, Feb 17, 2004 at 03:29:56PM +0100, Jeroen van Wolffelaar wrote:
> Hi,
>
> developer.php is quite slow sometimes, especially on people with much
> packages, you can notice delays, it even timeouts sometimes.
Thanks to Martin Michlmayr, I was able to test developer.php myself.
Find attached a patch with these changes:
- Fix all PHP warning that were in the script (but surpressed by the
server)
- Read bug information using a tree lookup (db2) rather than linear
search
- Fix a off-by-one mistake and a missing $, causing respectively a
warning and wrong comaintainer-behaviour (These were easy to catch
with warnings enabled)
- Add a 'Page generated in' item at the bottom
And a 'convert-bugs-to-db' script, to be run (after fixing paths)
immediately after bugs.txt is updated (new db is moved over old, no race
conditions).
Note: if for some reason qa.debian.org isn't running woody (where I
tested it on), you might need to change db2 to db3 or db4, I worked
around the current bug in php4 (but you do need an uptodate php4 for it
to work).
While with the current implementation the developer.php page for
[EMAIL PROTECTED] fails with a 30sec timelimit timeout, that same
page takes only 0.1 seconds with this updated implementation.
Test URL: http://jeroen.a-eskwadraat.nl/misc/qa/developer.php
(not updated with new info, unless someone can get me new extract &
bugs.txt files on the required intervals).
A developer.php page with freetext developer name still takes about
700ms constant time, I plan to look after that later, but currently other
things on my TODO list have more priority.
An improvement that shouldn't be hard, is making developer.php able to
generate a package-table based on other keys that the maintainer, like
based on task, or on a certain package with all his dependencies, stuff
like that. IMHO quite useful tracking down stuff.
--Jeroen
--
Jeroen van Wolffelaar
[EMAIL PROTECTED] (also for Jabber & MSN; ICQ: 33944357)
http://Jeroen.A-Eskwadraat.nl
Index: developer.wml
===================================================================
RCS file: /cvs/qa/wml/developer.wml,v
retrieving revision 1.49
diff -u -r1.49 developer.wml
--- developer.wml 20 Nov 2003 14:09:21 -0000 1.49
+++ developer.wml 18 Feb 2004 18:11:49 -0000
@@ -28,6 +28,7 @@
</style>
<?
+$time = microtime();
$branch = array ("main",
"contrib",
@@ -322,7 +323,7 @@
}
}
$global_count -= $result;
- }elseif(comaint == "only") {
+ }elseif($comaint == "only") {
$result = 0;
for ($i = 0; $i < strlen($packages); $i++) {
if ($packages[$i] == '#') {
@@ -410,18 +411,15 @@
$priority = $results[6];
$section = $results[7];
- $fp = fopen("$prefix/bugs.txt", "r");
- $bug_list = explode(" ", "0(0) 0(0) 0(0) 0(0)");
- $temp = preg_replace("/\+/", "_", $package);
- while(!feof($fp)) {
- $line = fgets($fp, "512");
- $line = preg_replace("/\+/", "_", $line);
-
if(preg_match("/^$temp\s+(\d+\(\d+\)\s\d+\(\d+\)\s\d+\(\d+\)\s\d+\(\d+\))/",
$line, $results)) {
- $bug_list = explode(" ", $results[1]);
- break;
- }
+ // make $bugsdb link persistent during this pagerequest, we don't lock, so
+ // this doesn't matter at all
+ static $bugsdb;
+ if (!$bugsdb) $bugsdb = dba_open("$prefix/bugs.db", 'r-', 'db2');
+
+ if (!($bug_list = dba_fetch($package, $bugsdb))) {
+ $bug_list = '0(0) 0(0) 0(0) 0(0)';
}
- fclose($fp);
+ $bug_list = explode(' ', $bug_list);
if(strcmp($version_stable, "-") == 0 || !debcheckavailable('stable',
$package)) {
$stable_dep = "<small>-</small>";
@@ -499,12 +497,12 @@
}
}
- if ($background == 1) {
+ if (@$background) {
print '<tr align="center">';
- $background=0;
+ $background=false;
}else{
print '<tr align="center" bgcolor="#dcdcdc">';
- $background=1;
+ $background=true;
}
?>
<td width="15%">
@@ -529,8 +527,8 @@
$bugs_all = "http://bugs.debian.org/cgi-bin/pkgreport.cgi?src=";.
rawurlencode($package);
$bug_count = $bug_list[0] + $bug_list[1] + $bug_list[2] + $bug_list[3];
$real_bug_count = 0;
- for ($i = 0; $i <= count($bug_list); $i++) {
- preg_match("/\d+\((\d+)\)/", $bug_list[$i], $count_array);
+ foreach ($bug_list as $bug_item) {
+ preg_match("/\d+\((\d+)\)/", $bug_item, $count_array);
$real_bug_count += $count_array[1];
}
if($bug_count == 0) { ?>
@@ -607,7 +605,7 @@
<td>
<small>
<?
- if($all == 1 || preg_match("/non-free/", $branch) ||
(strcmp($version_unstable, "-") == 0)) {
+ if(@$all || preg_match("/non-free/", $branch) || (strcmp($version_unstable,
"-") == 0)) {
print "-";
$all = 0;
}else{
@@ -670,10 +668,10 @@
<?
}
-if($comaint == "") {
+if([EMAIL PROTECTED]) {
$comaint = "yes";
}
-if($excuse != "") {
+if(@$excuse) {
$char = substr($excuse, 0, 1);
$excuse = preg_replace("/ /", "+", $excuse);
if(file_exists("$prefix/excuses/$char/$excuse")) {
@@ -683,7 +681,7 @@
}else{
print "<h2>No excuse for $excuse</h2>";
}
-}else if($gpg_key != "") {
+}else if (@$gpg_key) {
$contents = file("$prefix/extract");
$printed = 0;
@@ -715,7 +713,7 @@
}
printBackLink();
-}else if($login != "") {
+}else if(@$login) {
$fp = fopen("$prefix/extract", "r");
$contents = fread($fp, filesize ("$prefix/extract"));
@@ -753,7 +751,7 @@
}
fclose($fp);
printBackLink();
-}else if ($package != "") {
+}else if (@$package) {
$fp = fopen("$prefix/extract", "r");
$found = 0;
if(preg_match("/\+\+/", $package)) {
@@ -796,7 +794,7 @@
printBackLink();
}else{
print_form();
- if($all) {
+ if(@$all) {
print_all();
}else{
print_query_all();
@@ -818,5 +816,10 @@
General information: <? print $general; ?><br>
Bug information: <? print $bugs; ?><br>
General information is updated once a day.<br>
-Bug information is updated every 15 minutes.
+Bug information is updated every 15 minutes.<br>
+<?
+list($micro,$sec) = explode(' ', microtime());
+list($smicro,$ssec) = explode(' ', $time);
+echo "Time needed to generate page: ".($sec-$ssec+$micro-$smicro)."s";
+?>
</small></p>
#!/usr/bin/php4 -q
<?
$db = dba_open('bugs.db.new', 'n-', 'db2')
or die("Opening bugs.db failed");
$fp = fopen("bugs.txt", "r");
while ($line = fgets($fp, 512)) {
list($package, $bugdata) = preg_split('/\\s+/', $line, 2);
$bugdata = trim($bugdata);
dba_insert($package, $bugdata, $db)
or die("Insert for $package failed");
}
dba_optimize($db)
or die("Optimize failed");
dba_close($db);
rename('bugs.db.new', 'bugs.db');
