New submission from Andres Moreira <elkpich...@gmail.com>:
Hi,
I make a little change in the urlsplit function of urlparse module. And
the performance when I'm parsing a lot of urls was improved a bit.
In the best case was improved around a 20% percent.
Python version:
2.5.2 (r252:60911, Oct 5 2008, 19:29:17)
[GCC 4.3.2]
Here are the benchmarks:
#:~/tests$ python profile_urlparse.py
timing urlparse.urlparse():
[0.28006601333618164, 0.27513313293457031, 0.20408511161804199]
timing myurlparse.urlparse():
[0.11000704765319824, 0.10729002952575684, 0.10677695274353027]
#:~/tests$ python profile_urlparse2.py
timing urlparse.urlparse():
[0.28334403038024902, 0.27912592887878418, 0.15959692001342773]
timing myurlparse.urlparse():
[0.11277103424072266, 0.11163187026977539, 0.11175107955932617]
#:~/tests$ python profile_urlparse2.py
timing urlparse.urlparse():
[0.28750920295715332, 0.2779538631439209, 0.27816200256347656]
timing myurlparse.urlparse():
[0.25010085105895996, 0.11236691474914551, 0.11198592185974121]
#-- Paste here the profiling code -----------
#-- Please rename the file as : profile_urlparse.py
urls = [
"http://www.notonthehighstreet.com/boxwood/product/dotty_picture_frames";,
"http://www.fancylighting.com/acatalog/Petrushka_Bronze.html";,
"http://cgi.ebay.co.uk/3-LITRE-SUNNEX-STAINLESS-STEEL-TEAPOT-COFFEE-POT_W0QQitemZ160230173283QQcategoryZ122942QQcmdZViewItem";,
"http://retail.ictc.co.uk/acatalog/Online_Catalogue__Homegrown_184.html";,
"http://www.amazon.co.uk/Big-Mog-Tape-Judith-Kerr/dp/0001025252?SubscriptionId=0QE3E4F7T4Q5DCSKG202&tag=ws&linkCode=xm2&camp=2025&creative=165953&creativeASIN=0001025252";,
"http://www.dinerstore.co.uk/acatalog/copy_of_Bedford_Oak_Table_and_Six.html";,
"http://www.panik-design.co.uk/acatalog/Iittala_-_A_Citterio_-_Citterio_98_Cutlery_24pcs_.html";,
"http://www.johnlewis.com/230544027/Product.aspx";,
"http://cgi.ebay.co.uk/Damask-Black-Pink-Cream-Large-Modern-Rugs-120x170cm_W0QQitemZ400021540458QQcategoryZ57237QQcmdZViewItem";,
"http://www.amazon.co.uk/Nikon-50Mm-F1-2-Nikkor-Lens/dp/B00009R95Y?SubscriptionId=0QE3E4F7T4Q5DCSKG202&tag=ws&linkCode=xm2&camp=2025&creative=165953&creativeASIN=B00009R95Y";,
"http://www.amazon.co.uk/Storeys-Guide-Raising-Llamas-Birutta/dp/1580173284?SubscriptionId=0QE3E4F7T4Q5DCSKG202&tag=ws&linkCode=xm2&camp=2025&creative=165953&creativeASIN=1580173284";,
"http://www.24electric.com/detail.php?ProdID=46519663";,
"http://cgi.ebay.co.uk/Antique-Pine-Midi-Sleeper-Childrens-Bed-VGC_W0QQitemZ280285887713QQcategoryZ122763QQcmdZViewItem";,
"http://www.johnlewis.com/230421907/Product.aspx";,
"http://cgi.ebay.co.uk/WICKER-PLACE-DINNER-MATS-X6-IN-A-WICKER-BASKET-WITH-LID_W0QQitemZ350141392941QQcategoryZ20660QQcmdZViewItem";,
"http://www.trueshopping.co.uk/product/Draper_1_4_Square_Drive_Reversible_Ratchet/3495/43235.html";,
"http://www.amazon.co.uk/Transcend-TS128MIB6986-128MB-Module/dp/B000HCO61K?SubscriptionId=0QE3E4F7T4Q5DCSKG202&tag=ws&linkCode=xm2&camp=2025&creative=165953&creativeASIN=B000HCO61K";,
"http://www.unitedinteriors.co.uk/regency-pine-hi-fi-cabinet-4741-p.asp";,
"http://www.amazon.co.uk/BATTERY-CAMCORDER-DCR-DVD602-DCR-DVD602E-DCRDVD602/dp/B0017UM1OU?SubscriptionId=0QE3E4F7T4Q5DCSKG202&tag=ws&linkCode=xm2&camp=2025&creative=165953&creativeASIN=B0017UM1OU";,
"http://www.tooled-up.com/Product.asp?PID=145138";,
"http://www.dinerstore.co.uk/acatalog/copy_of_Bedford_Oak_Table_and_Six.html";,
"http://www.panik-design.co.uk/acatalog/Iittala_-_A_Citterio_-_Citterio_98_Cutlery_24pcs_.html";,
"http://www.johnlewis.com/230544027/Product.aspx";,
"http://cgi.ebay.co.uk/Damask-Black-Pink-Cream-Large-Modern-Rugs-120x170cm_W0QQitemZ400021540458QQcategoryZ57237QQcmdZViewItem";,
"http://www.amazon.co.uk/Nikon-50Mm-F1-2-Nikkor-Lens/dp/B00009R95Y?SubscriptionId=0QE3E4F7T4Q5DCSKG202&tag=ws&linkCode=xm2&camp=2025&creative=165953&creativeASIN=B00009R95Y";,
"http://www.amazon.co.uk/Storeys-Guide-Raising-Llamas-Birutta/dp/1580173284?SubscriptionId=0QE3E4F7T4Q5DCSKG202&tag=ws&linkCode=xm2&camp=2025&creative=165953&creativeASIN=1580173284";,
"http://www.24electric.com/detail.php?ProdID=46519663";,
"http://cgi.ebay.co.uk/Antique-Pine-Midi-Sleeper-Childrens-Bed-VGC_W0QQitemZ280285887713QQcategoryZ122763QQcmdZViewItem";,
"http://www.johnlewis.com/230421907/Product.aspx";,
"http://cgi.ebay.co.uk/WICKER-PLACE-DINNER-MATS-X6-IN-A-WICKER-BASKET-WITH-LID_W0QQitemZ350141392941QQcategoryZ20660QQcmdZViewItem";,
"http://www.trueshopping.co.uk/product/Draper_1_4_Square_Drive_Reversible_Ratchet/3495/43235.html";,
]
if __name__ == '__main__':
import sys
import timeit
if len(sys.argv) > 1:
times = int(sys.argv[1])
else:
times = 1000
t = timeit.Timer("[urlparse.urlparse(u) for u in urls]",
"from profile_urlparse import urls; import urlparse")
print "timing urlparse.urlparse():"
print " ", t.repeat(3, times)
t = timeit.Timer("[myurlparse.urlparse(u) for u in urls]",
"from profile_urlparse import urls; import myurlparse")
print "timing myurlparse.urlparse():"
print " ", t.repeat(3, times)
#--- End of profile code ----------------------------------------
----------
components: Library (Lib)
files: urlparse.py
messages: 79752
nosy: andrix
severity: normal
status: open
title: Little improvement on urlparse module, urlparse function.
type: performance
versions: Python 2.5
Added file: http://bugs.python.org/file12720/urlparse.py
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue4932>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
