[ python-Bugs-1610654 ] cgi.py multipart/form-data
Bugs item #1610654, was opened at 2006-12-07 09:18
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1610654&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Performance
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Chui Tey (teyc)
Assigned to: Nobody/Anonymous (nobody)
Summary: cgi.py multipart/form-data
Initial Comment:
Uploading large binary files using multipart/form-data can be very inefficient
because LF character may occur too frequently, resulting in the
read_line_to_outer_boundary looping too many times.
*** cgi.py.Py24 Thu Dec 7 18:46:13 2006
--- cgi.py Thu Dec 7 16:38:04 2006
***
*** 707,713
last = next + "--"
delim = ""
while 1:
! line = self.fp.readline()
if not line:
self.done = -1
break
--- 703,709
last = next + "--"
delim = ""
while 1:
! line = self.fp_readline()
if not line:
self.done = -1
break
***
*** 729,734
--- 730,753
delim = ""
self.__write(odelim + line)
+ def fp_readline(self):
+
+ tell = self.fp.tell()
+ buffer = self.fp.read(1 << 17)
+ parts = buffer.split("\n")
+ retlst = []
+ for part in parts:
+ if part.startswith("--"):
+ if retlst:
+ retval = "\n".join(retlst) + "\n"
+ else:
+ retval = part + "\n"
+ self.fp.seek(tell + len(retval))
+ return retval
+ else:
+ retlst.append(part)
+ return buffer
+
def skip_lines(self):
"""Internal: skip lines until outer boundary if defined."""
if not self.outerboundary or self.done:
The patch reads the file in larger increments. For my test file of 138 Mb, it
reduced parsing time from 168 seconds to 19 seconds.
# test script
import cgi
import cgi
import os
import profile
import stat
def run():
filename = 'body.txt'
size = os.stat(filename)[stat.ST_SIZE]
fp = open(filename,'rb')
environ = {}
environ["CONTENT_TYPE"] = open('content_type.txt','rb').read()
environ["REQUEST_METHOD"] = "POST"
environ["CONTENT_LENGTH"] = str(size)
fieldstorage = cgi.FieldStorage(fp, None, environ=environ)
return fieldstorage
import hotshot, hotshot.stats
import time
if 1:
t1 = time.time()
prof = hotshot.Profile("bug1718.prof")
# hotshot profiler will crash with the
# patch applied on windows xp
#prof_results = prof.runcall(run)
prof_results = run()
prof.close()
t2 = time.time()
print t2-t1
if 0:
for key in prof_results.keys():
if len(prof_results[key].value)> 100:
print key, prof_results[key].value[:80] + "..."
else:
print key, prof_results[key]
content_type.txt
multipart/form-data; boundary=--ThIs_Is_tHe_bouNdaRY_$
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1610654&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-1592899 ] "".translate() docs should mention string.maketrans()
Feature Requests item #1592899, was opened at 2006-11-08 20:23 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1592899&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: Python 2.6 >Status: Closed >Resolution: Fixed Priority: 5 Private: No Submitted By: Ori Avtalion (salty-horse) Assigned to: Raymond Hettinger (rhettinger) Summary: "".translate() docs should mention string.maketrans() Initial Comment: The translate() documentation at http://docs.python.org/lib/string-methods.html#l2h-268 should mention the string.maketrans helper function. maketrans also mentions "regex.compile" - that should probably be "re.compile" (although it's less readable). re.compile should mention maketrans as well. -- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-07 09:30 Message: Logged In: YES user_id=849994 Originator: NO Cleared up both items in rev. 52951, 52952 (2.5). -- Comment By: Fredrik Lundh (effbot) Date: 2006-12-04 17:42 Message: Logged In: YES user_id=38376 Originator: NO Aha. The regex documentation says: compile (pattern[, translate]) /.../ The optional argument translate, if present, must be a 256-character string indicating how characters (both of the pattern and of the strings to be matched) are translated before comparing them; the i-th element of the string gives the translation for the character with ASCII code i. This can be used to implement case-insensitive matching; see the casefold data item below. which means that it was indeed useful for regex.compile. afaik, re.compile doesn't support arbitrary mappings, so that reference should be removed. -- Comment By: Ori Avtalion (salty-horse) Date: 2006-12-04 17:08 Message: Logged In: YES user_id=854801 Originator: YES >"re.compile should mention maketrans as well" >why? The maketrans doc says: "Return a translation table suitable for passing to translate() or regex.compile()" I don't have any use case for how it's useful to regex.compile. Maybe there is none, and the mention can be removed. -- Comment By: Fredrik Lundh (effbot) Date: 2006-12-04 09:20 Message: Logged In: YES user_id=38376 Originator: NO "re.compile should mention maketrans as well" why? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1592899&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1611131 ] \b in unicode regex gives strange results
Bugs item #1611131, was opened at 2006-12-07 23:44 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1611131&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Regular Expressions Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: akaihola (akaihola) Assigned to: Gustavo Niemeyer (niemeyer) Summary: \b in unicode regex gives strange results Initial Comment: The problem: This doesn't give a match: >>> re.match(r'ä\b', 'ä ', re.UNICODE) This works ok and gives a match: >>> re.match(r'.\b', 'ä ', re.UNICODE) Both of these work as well: >>> re.match(r'a\b', 'a ', re.UNICODE) >>> re.match(r'.\b', 'a ', re.UNICODE) Docs say \b is defined as an empty string between \w and \W. These do match accordingly: >>> re.match(r'\w', 'ä', re.UNICODE) >>> re.match(r'\w', 'a', re.UNICODE) >>> re.match(r'\W', ' ', re.UNICODE) So something strange happens in my first example, and I can't help but assume it's a bug. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1611131&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1611131 ] \b in unicode regex gives strange results
Bugs item #1611131, was opened at 2006-12-07 23:44 Message generated for change (Comment added) made by akaihola You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1611131&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Regular Expressions Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: akaihola (akaihola) Assigned to: Gustavo Niemeyer (niemeyer) Summary: \b in unicode regex gives strange results Initial Comment: The problem: This doesn't give a match: >>> re.match(r'ä\b', 'ä ', re.UNICODE) This works ok and gives a match: >>> re.match(r'.\b', 'ä ', re.UNICODE) Both of these work as well: >>> re.match(r'a\b', 'a ', re.UNICODE) >>> re.match(r'.\b', 'a ', re.UNICODE) Docs say \b is defined as an empty string between \w and \W. These do match accordingly: >>> re.match(r'\w', 'ä', re.UNICODE) >>> re.match(r'\w', 'a', re.UNICODE) >>> re.match(r'\W', ' ', re.UNICODE) So something strange happens in my first example, and I can't help but assume it's a bug. -- >Comment By: akaihola (akaihola) Date: 2006-12-08 00:18 Message: Logged In: YES user_id=1432932 Originator: YES As a work-around I currently use a regex like r'ä(?=\W)'. Seems to work ok. Also, the \b problem doesn't seem to exist in the \W\w case, i.e. at the beginning of words. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1611131&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1611154 ] os.path.exists("file/") failure on Solaris 9
Bugs item #1611154, was opened at 2006-12-07 22:25
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1611154&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Paul Eggert (eggert)
Assigned to: Nobody/Anonymous (nobody)
Summary: os.path.exists("file/") failure on Solaris 9
Initial Comment:
Solaris 9 and earlier fail to conform to POSIX, in that
stat("FILE/") succeeds even when FILE is not a directory.
POSIX says that in this case it should fail. This problem causes
os.path.exists("FILE/") to succeed when it should fail, which makes it harder
to write portable Python code. One of my students ran into this problem when
doing a Django-based project: his code ran fine on his Linux box, but failed
when he attempted to run it on the Solaris 8 server that is the standard
platform for our students.
To reproduce the problem, on Solaris 8 (or 9):
$ touch file
$ ls -l file
-rw-rw-r-- 1 eggert csfac 0 Dec 7 14:19 file
$ python
Python 2.5 (r25:51908, Dec 7 2006, 13:14:10)
[GCC 4.1.1] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path.exists("file/")
True
It should be False.
I'll attach a patch that works around the problem at
run-time. If you prefer something that tests for it
at compile time please let me know.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1611154&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1610485 ] GUI for Python 2.3, 2.4, and 2.5 is very sluggish
Bugs item #1610485, was opened at 2006-12-06 18:44 Message generated for change (Comment added) made by g4rlik You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1610485&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: Python 2.5 >Status: Open Resolution: Works For Me Priority: 5 Private: No Submitted By: g4rlik (g4rlik) Assigned to: Kurt B. Kaiser (kbk) Summary: GUI for Python 2.3, 2.4, and 2.5 is very sluggish Initial Comment: The GUI for Python versions 2.3, 2.4, and 2.5 are very sluggish. When I type in them, or simply move them around my desktop, they are very slow. Someone helping me from [EMAIL PROTECTED] was able to guess that the reason the GUI is slow for me is because of the subprocesses running, and he was right. To cope with this problem, I created an idle.txt file, and added this to the first line of it: c:\python25\python c:\python25\Lib\idlelib\idle.py -n After that I saved the extension to .bat and now when I run that file, the Python GUI opens without any subprocesses running and I have no problem. However, I'd still really like to know how I could fix this problem even more. The GUIs for Python version 2.2 and below run fine for me. I am using Windows XP Home Edition Service Pack 2. -- >Comment By: g4rlik (g4rlik) Date: 2006-12-07 19:14 Message: Logged In: YES user_id=1662589 Originator: YES I ran some programs to get rid of adaware and spyware and then tried to run the GUI for 2.5. I still have the problem. After that I unplugged my router (which has the internal firewall) and attempted to test the GUI's speed again. It was still sluggish. When running the GUI and looking in Task Manager, it uses <1% of my CPU's power. By the way, the specs for my computer are: AMD Athlon XP 2800+ (2.08ghz) 1 gig RAM I don't have an amazing rig, but that should do more than fine for running Python. I don't believe my CPU and RAM are my problems. Oh well, I don't think there is much more looking into this that I can do. I'll either have to live with the sluggishness or run Python 2.2. Thanks. -- Comment By: Kurt B. Kaiser (kbk) Date: 2006-12-07 00:53 Message: Logged In: YES user_id=149084 Originator: NO I have not seen anything like this myself on W2K or WinXP, nor have I heard of something like this before. We switched to the subprocess version of IDLE at 2.3. When printing a mass of text to the Shell window, IDLE is about 30% slower when using the subprocess, but that doesn't sound like what you are reporting. When I move IDLE's windows around there is no perceptible delay, nor can I detect any slowness when typing. (The 30% isn't involved when doing that). I can run i = 1 while True: i +=1 print i and the GUI responds quickly even though the cpu is near 100%. If the solution works for you, fine, but it seems likely to me that there is something misconfigured with your Windows installation or that it is compromised by spyware or otherwise overloaded or having a hardware problem. I'm setting this bug as pending, works for me unless you can come up with something more definitive. What does your Task Manager show for CPU utilization when you are having this problem? It seems that something is hogging your cpu or you have a very slow computer. What's the clock rate and memory size? If you are running an internal firewall, try disconnecting from the net and turn off the firewall temporarily to see if the problem is there. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1610485&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1610485 ] GUI for Python 2.3, 2.4, and 2.5 is very sluggish
Bugs item #1610485, was opened at 2006-12-06 18:44 Message generated for change (Comment added) made by kbk You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1610485&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: Python 2.5 Status: Open Resolution: Works For Me Priority: 5 Private: No Submitted By: g4rlik (g4rlik) Assigned to: Kurt B. Kaiser (kbk) Summary: GUI for Python 2.3, 2.4, and 2.5 is very sluggish Initial Comment: The GUI for Python versions 2.3, 2.4, and 2.5 are very sluggish. When I type in them, or simply move them around my desktop, they are very slow. Someone helping me from [EMAIL PROTECTED] was able to guess that the reason the GUI is slow for me is because of the subprocesses running, and he was right. To cope with this problem, I created an idle.txt file, and added this to the first line of it: c:\python25\python c:\python25\Lib\idlelib\idle.py -n After that I saved the extension to .bat and now when I run that file, the Python GUI opens without any subprocesses running and I have no problem. However, I'd still really like to know how I could fix this problem even more. The GUIs for Python version 2.2 and below run fine for me. I am using Windows XP Home Edition Service Pack 2. -- >Comment By: Kurt B. Kaiser (kbk) Date: 2006-12-08 00:13 Message: Logged In: YES user_id=149084 Originator: NO Your system is powerful enough, by an order of magnitude :-) My W2K system is about 400 Mhz. It shows no slowdown with 2.3 - 2.5. If your system is slow only with the subprocess, there must be something about using the socket interface that is problematic. I don't know much about XP Home, but I vaguely recollect hearing about some difficulties with it. You might try upgrading to Win XP Pro, or install Linux. -- Comment By: g4rlik (g4rlik) Date: 2006-12-07 19:14 Message: Logged In: YES user_id=1662589 Originator: YES I ran some programs to get rid of adaware and spyware and then tried to run the GUI for 2.5. I still have the problem. After that I unplugged my router (which has the internal firewall) and attempted to test the GUI's speed again. It was still sluggish. When running the GUI and looking in Task Manager, it uses <1% of my CPU's power. By the way, the specs for my computer are: AMD Athlon XP 2800+ (2.08ghz) 1 gig RAM I don't have an amazing rig, but that should do more than fine for running Python. I don't believe my CPU and RAM are my problems. Oh well, I don't think there is much more looking into this that I can do. I'll either have to live with the sluggishness or run Python 2.2. Thanks. -- Comment By: Kurt B. Kaiser (kbk) Date: 2006-12-07 00:53 Message: Logged In: YES user_id=149084 Originator: NO I have not seen anything like this myself on W2K or WinXP, nor have I heard of something like this before. We switched to the subprocess version of IDLE at 2.3. When printing a mass of text to the Shell window, IDLE is about 30% slower when using the subprocess, but that doesn't sound like what you are reporting. When I move IDLE's windows around there is no perceptible delay, nor can I detect any slowness when typing. (The 30% isn't involved when doing that). I can run i = 1 while True: i +=1 print i and the GUI responds quickly even though the cpu is near 100%. If the solution works for you, fine, but it seems likely to me that there is something misconfigured with your Windows installation or that it is compromised by spyware or otherwise overloaded or having a hardware problem. I'm setting this bug as pending, works for me unless you can come up with something more definitive. What does your Task Manager show for CPU utilization when you are having this problem? It seems that something is hogging your cpu or you have a very slow computer. What's the clock rate and memory size? If you are running an internal firewall, try disconnecting from the net and turn off the firewall temporarily to see if the problem is there. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1610485&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
