XSLT speed comparisons
Hi, I'm from an ASP.NET background an am considering making the switch
to Python. I decided to develop my next project in tandem to test the
waters and everything is working well, loving the language, etc.
What I've got is:
two websites, one in ASP.NET v2 and one in Python 2.5 (using 4suite for
XML/XSLT)
both on the same box (Windows Server 2003)
both using the same XML, XSLT, CSS
The problem is, the Python version is (at a guess) about three times
slower than the ASP one. I'm very new to the language and it's likely
that I'm doing something wrong here:
from os import environ
from Ft.Lib.Uri import OsPathToUri
from Ft.Xml import InputSource
from Ft.Xml.Xslt import Processor
def buildPage():
try:
xsluri = OsPathToUri('xsl/plainpage.xsl')
xmluri = OsPathToUri('website.xml')
xsl = InputSource.DefaultFactory.fromUri(xsluri)
xml = InputSource.DefaultFactory.fromUri(xmluri)
proc = Processor.Processor()
proc.appendStylesheet(xsl)
params = {"url":environ['QUERY_STRING'].split("=")[1]}
for i, v in enumerate(environ['QUERY_STRING'].split("/")[1:]):
params["selected_section%s" % (i + 1)] = "/" + v
return proc.run(xml, topLevelParams=params)
except:
return "Error blah blah"
print "Content-Type: text/html\n\n"
print buildPage()
You can compare the development sites here:
asp: http://consultum.pointy.co.nz/about/team
python: http://python.pointy.co.nz/about/team
Cheers!
--
http://mail.python.org/mailman/listinfo/python-list
Re: XSLT speed comparisons
Ross Ridge wrote: > It could just be that 4suite is slower than MSXML. If so, you can use > MSXML in Python if you want. You'll need to install the Python for > Windows extensions. Something like this: Thanks for that Ross. That would make sense, I'd read somewhere that the 4suite code was a little slower but wanted to be sure it wasn't just something I'd done wrong. I've experimented with the code you supplied and after installing the necessaries have run into a brick wall with a series of errors. The errors can be seen at http://python.pointy.co.nz/test (I'm leaving the existing, slower version running at the moment for the rest of the site). I've got to get back on with some work but will look further into this tonight. Thanks for your help! I really appreciate it. -- http://mail.python.org/mailman/listinfo/python-list
Re: XSLT speed comparisons
Ross Ridge wrote:
> Hmm... it seems that you don't have MSXML 4.0 installed on your
> machine. I missed the fact that you're using ASP.NET, so your ASP code
> probably is probably using the .NET XML implementation instead of
> MSXML. In that case, another alternative might be to use IronPython
> and just translate your ASP script into Python.
>
> Ross Ridge
Sorted!
I installed msxml4 and then struggled for an hour or so with an
encoding error (UnicodeEncodeError: 'ascii' codec etc) which was
fixed by altering your code from:
return proc.output --> return proc.output.encode('utf-8')
The performance of MSXML over 4suite is substantial.
4suite: http://python.pointy.co.nz/test = 2.5s
MSXML: http://python.pointy.co.nz/test_msxml = 1.1s
I'd like to eventually break all ties with MS at some stage. It'll be
interesting to test this performance on a Linux server.
Thank you for your help Ross.
--
http://mail.python.org/mailman/listinfo/python-list
Re: XSLT speed comparisons
Ross Ridge wrote:
> Hmm... it seems that you don't have MSXML 4.0 installed on your
> machine. I missed the fact that you're using ASP.NET, so your ASP code
> probably is probably using the .NET XML implementation instead of
> MSXML. In that case, another alternative might be to use IronPython
> and just translate your ASP script into Python.
Sorted!
I installed msxml4 and then struggled for an hour or so with an
encoding error (UnicodeEncodeError: 'ascii' codec etc) which was
fixed by altering your code from:
return proc.output --> return proc.output.encode('utf-8')
The performance of MSXML over 4suite is substantial.
4suite: http://python.pointy.co.nz/test = 2.5s
MSXML: http://python.pointy.co.nz/test_msxml = 1.1s
I'd like to eventually break all ties with MS at some stage. It'll be
interesting to test this performance on a Linux server.
Thank you for your help.
Damian
--
http://mail.python.org/mailman/listinfo/python-list
Re: XSLT speed comparisons
Sorted!
I installed msxml4 and then struggled for an hour or so with an
encoding error (UnicodeEncodeError: 'ascii' codec etc) which was
fixed by altering your code from:
return proc.output --> return proc.output.encode('utf-8')
The performance of MSXML over 4suite is substantial.
4suite: http://python.pointy.co.nz/test = 2.5s
MSXML: http://python.pointy.co.nz/test_msxml = 1.1s
I'd like to eventually break all ties with MS at some stage. It'll be
interesting to test this performance on a Linux server.
Thank you for your help.
Damian
--
http://mail.python.org/mailman/listinfo/python-list
Re: XSLT speed comparisons
Sorry about the multiple posts folks. I suspect it was the "FasterFox" FireFox extension I installed yesterday tricking me. I had a brief look at libxml(?) on my Ubuntu machine but haven't run it on the server. I'll look into pyrxp Larry. I have to say I'm struggling a little with the discoverability and documentation side of things with Python. I realise that discoverability is purported to be one of its strong sides but coming from the Visual Studio IDE where Intellisense looks after everything as you are typing and you can see exactly what methods are available to what class and what variables are required and why what I've seen so far is not that flash. I've installed Eclipse with Pydev (very impressive) on my Linux box and am using IDLE on Windows and it could just be my lack of familiarity that is letting me down. Any other IDE recommendations? I'd be keen to test pyrxp and libxslt but may need help with the code - I spent literally hours yesterday trying to make a 20-line bit of code work. To make things worse I started with 4suite in Ubuntu and it refused to work with an error about not being able to find default.cat or something. Googled for hours with no luck. That said, I really want to make the switch and so far Python looks to be the best choice. Cheers Damian -- http://mail.python.org/mailman/listinfo/python-list
Re: XSLT speed comparisons
A, thanks for that, I've been searching the documentation and it only briefly mentions XSLT but it sounds like a half-arsed attempt. -- http://mail.python.org/mailman/listinfo/python-list
Support both asyncio and synchronous callers
Hi. I'm the author of Stem, Tor's python library [1]. Recently we
migrated to asyncio, but desire to still be usable by synchronous
callers.
We wrote a mixin [2][3] that transparently makes any class usable by
both asyncio and synchronous callers...
==
import asyncio
import stem.util.asyncio
class Example(stem.util.asyncio.Synchronous):
async def hello(self):
return 'hello'
def sync_demo():
instance = Example()
print('%s from a synchronous context' % instance.hello())
instance.stop()
async def async_demo():
instance = Example()
print('%s from an asynchronous context' % await instance.hello())
instance.stop()
sync_demo()
asyncio.run(async_demo())
==
Is there a better way to approach this?
Our wider ecosystem seem to be forking networking libraries into
sync/async variants, with requests [4] and AIOHTTP [5] as the most
prominent. Are there any libraries out there that demonstrate good
support for both kinds of callers without duplicating their API?
Thanks! -Damian
PS. I apologize if this email gets duplicated. Initially I used a
different email address which Mailman seemed to decline.
[1] https://stem.torproject.org/
[2] https://gitweb.torproject.org/stem.git/tree/stem/util/asyncio.py
[3] https://gitweb.torproject.org/stem.git/tree/test/unit/util/asyncio.py
[4] https://requests.readthedocs.io/en/master/
[5] https://docs.aiohttp.org/en/latest/index.html
--
https://mail.python.org/mailman/listinfo/python-list
Curses Blank Background
Does anyone know how to instruct the Python curses bindings to leave the background alone (use the default terminal background)? I'm interested in keeping my semi-transparent background which curses can't replicate. This question was raised on this list before ( http://mail.python.org/pipermail/python-list/2001-July/094581.html) but it never got a reply. From the ncurses man page it looks like this functionality would be mapped to -1 but the Python curses module uses -1 for ERR. I'm new to curses so my apologies if I'm missing something obvious. Cheers! -Damian -- http://mail.python.org/mailman/listinfo/python-list
Python 3.0 Curses Unicode
Hi, I've switched to Python 3.0 for a new Japanese vocab quizzing
application due to its much improved Unicode support. However, I'm running
into an issue with displaying Unicode characters via curses. In Python 2.x a
simple hello-world looks like:
#!/usr/bin/python
# coding=UTF-8
import curses
import locale
locale.setlocale(locale.LC_ALL,"")
def doStuff(stdscr):
message = u"hello わたし!"
stdscr.addstr(0, 0, message.encode("utf-8"), curses.A_BLINK)
stdscr.getch() # pauses until a key's hit
curses.wrapper(doStuff)
This works. However, when I try to come up with an equivalent for Python
3.0:
#!/usr/bin/python
import curses
import locale
locale.setlocale(locale.LC_ALL,"")
def doStuff(stdscr):
message = "hello わたし!"
stdscr.addstr(0, 0, message, curses.A_BLINK)
stdscr.getch() # pauses until a key's hit
curses.wrapper(doStuff)
It fails (printing gibberish to the console). Anyone have a clue what I'm
doing wrong? Thanks! -Damian
PS. Is the "# coding=UTF-8" header meaningless in Python 3.0? Also, is
"locale.setlocale(locale.LC_ALL,"")" still necessary for getting curses to
provide Unicode support?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 3.0 Curses Unicode
It seems as if the curses module in Python 3.0 isn't respecting the system's
preferred encoding (utf-8) which was set via:
locale.setlocale(locale.LC_ALL, '')
The purpose of this was described at the top of '
http://docs.python.org/dev/3.0/library/curses.html#module-curses'. The
getlocale function is reporting the proper values ('en_US', 'UTF8') but
addstr is clearly not treating it as Unicode - is this a bug? -Damian
2008/12/28 Damian Johnson
> Hi, I've switched to Python 3.0 for a new Japanese vocab quizzing
> application due to its much improved Unicode support. However, I'm running
> into an issue with displaying Unicode characters via curses. In Python 2.x a
> simple hello-world looks like:
>
> #!/usr/bin/python
> # coding=UTF-8
>
> import curses
> import locale
>
> locale.setlocale(locale.LC_ALL,"")
>
> def doStuff(stdscr):
> message = u"hello わたし!"
> stdscr.addstr(0, 0, message.encode("utf-8"), curses.A_BLINK)
> stdscr.getch() # pauses until a key's hit
>
> curses.wrapper(doStuff)
>
> This works. However, when I try to come up with an equivalent for Python
> 3.0:
>
> #!/usr/bin/python
>
> import curses
> import locale
>
> locale.setlocale(locale.LC_ALL,"")
>
> def doStuff(stdscr):
> message = "hello わたし!"
> stdscr.addstr(0, 0, message, curses.A_BLINK)
> stdscr.getch() # pauses until a key's hit
>
> curses.wrapper(doStuff)
>
> It fails (printing gibberish to the console). Anyone have a clue what I'm
> doing wrong? Thanks! -Damian
>
> PS. Is the "# coding=UTF-8" header meaningless in Python 3.0? Also, is
> "locale.setlocale(locale.LC_ALL,"")" still necessary for getting curses to
> provide Unicode support?
>
>
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 3.0 Curses Unicode
Just resolved the issue (turned out to be an issue with linked ncurses
libraries). If others run into this discussion of the solution can be found
at:
http://bugs.python.org/issue4787
Cheers! -Damian
On Mon, Dec 29, 2008 at 10:30 PM, Damian Johnson wrote:
> It seems as if the curses module in Python 3.0 isn't respecting the
> system's preferred encoding (utf-8) which was set via:
> locale.setlocale(locale.LC_ALL, '')
>
> The purpose of this was described at the top of '
> http://docs.python.org/dev/3.0/library/curses.html#module-curses'. The
> getlocale function is reporting the proper values ('en_US', 'UTF8') but
> addstr is clearly not treating it as Unicode - is this a bug? -Damian
>
> 2008/12/28 Damian Johnson
>
> Hi, I've switched to Python 3.0 for a new Japanese vocab quizzing
>> application due to its much improved Unicode support. However, I'm running
>> into an issue with displaying Unicode characters via curses. In Python 2.x a
>> simple hello-world looks like:
>>
>> #!/usr/bin/python
>> # coding=UTF-8
>>
>> import curses
>> import locale
>>
>> locale.setlocale(locale.LC_ALL,"")
>>
>> def doStuff(stdscr):
>> message = u"hello わたし!"
>> stdscr.addstr(0, 0, message.encode("utf-8"), curses.A_BLINK)
>> stdscr.getch() # pauses until a key's hit
>>
>> curses.wrapper(doStuff)
>>
>> This works. However, when I try to come up with an equivalent for Python
>> 3.0:
>>
>> #!/usr/bin/python
>>
>> import curses
>> import locale
>>
>> locale.setlocale(locale.LC_ALL,"")
>>
>> def doStuff(stdscr):
>> message = "hello わたし!"
>> stdscr.addstr(0, 0, message, curses.A_BLINK)
>> stdscr.getch() # pauses until a key's hit
>>
>> curses.wrapper(doStuff)
>>
>> It fails (printing gibberish to the console). Anyone have a clue what I'm
>> doing wrong? Thanks! -Damian
>>
>> PS. Is the "# coding=UTF-8" header meaningless in Python 3.0? Also, is
>> "locale.setlocale(locale.LC_ALL,"")" still necessary for getting curses to
>> provide Unicode support?
>>
>>
>
--
http://mail.python.org/mailman/listinfo/python-list
Unicode equality from raw_input
Hi, when getting text via the raw_input method it's always a string (even if
it contains non-ASCII characters). The problem lies in that whenever I try
to check equality against a Unicode string it fails. I've tried using the
unicode method to 'cast' the string to the Unicode type but this throws an
exception:
>>> a = raw_input("text: ")
text: おはよう
>>> b = u"おはよう"
>>> a == b
__main__:1: UnicodeWarning: Unicode equal comparison failed to convert both
arguments to Unicode - interpreting them as being unequal
False
>>> type(a)
>>> type(b)
>>> unicode(a)
Traceback (most recent call last):
File "", line 1, in
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 0:
ordinal not in range(128)
>>> str(b)
Traceback (most recent call last):
File "", line 1, in
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-3:
ordinal not in range(128)
After a couple hours of hair pulling I think it's about time to admit
defeat. Any help would be appreciated! -Damian
--
http://mail.python.org/mailman/listinfo/python-list
Curses Subwin Resizing
Hi, I've been trying to make a curses application that's resilient to having the terminal resized (like 'top'). I've managed to make it capable of adjusting when the screen's width is changed (using getmaxyx) but changing the height is proving a little trickier. I'm using subwindows so I can selectively clear and redraw portions of the screen, however whenever the screen's height is reduced it permanently shrinks (and moves) displaced subwindows, so it can't recover when the screen's enlarged again. I've been looking for hours for any of the following that would fix the problem... ... the ability to move and resize subwindows so I can restore them. ... the ability to delete subwindows so I can just receate them. ... the ability to selectively clear a rectangular region so I don't need subwindows at all (I've spotted character deletion, but not region and it seems kinda funky to tell curses to clear each cell individually). This seems like it's probably a common problem but I haven't spotted a solution. Any help would be appreciated! -Damian -- http://mail.python.org/mailman/listinfo/python-list
Detecting Remaining Thread
Hi, I've been trying to track down a familiar concurrency problem without any success: Unhandled exception in thread started by Error in sys.excepthook: Original exception was: I realize that this is due to a background thread still being alive and kicking when the application terminates (ie, a missing join() on a helper thread). However, threading.enumerate() is reporting that nothing except for the main thread is running at that point: 1/19/2011 09:30:41 [INFO] Arm is shutting down. Remaining threads: [<_MainThread(MainThread, started -1217079616)>] What other methods are there for troubleshooting this sort of issue? I've triple checked that my threads are daemons and joined when quitting but I must be missing something... As would be expected of a timing issue, this happens intermittently (1/5 of the time) and goes away if I add a short sleep at the end. Any help would be much appreciated. Thanks! -Damian -- http://mail.python.org/mailman/listinfo/python-list
Re: Detecting Remaining Thread
Disregard, I figured it out. It turns out that the threads spawned by "thread.start_new_thread" are unreported by threading.enumerate. Cheers! -Damian On Wed, Jan 19, 2011 at 9:40 AM, Damian Johnson wrote: > Hi, I've been trying to track down a familiar concurrency problem > without any success: > Unhandled exception in thread started by > Error in sys.excepthook: > > Original exception was: > > I realize that this is due to a background thread still being alive > and kicking when the application terminates (ie, a missing join() on a > helper thread). However, threading.enumerate() is reporting that > nothing except for the main thread is running at that point: > 1/19/2011 09:30:41 [INFO] Arm is shutting down. Remaining threads: > [<_MainThread(MainThread, started -1217079616)>] > > What other methods are there for troubleshooting this sort of issue? > I've triple checked that my threads are daemons and joined when > quitting but I must be missing something... > > As would be expected of a timing issue, this happens intermittently > (1/5 of the time) and goes away if I add a short sleep at the end. Any > help would be much appreciated. Thanks! -Damian > -- http://mail.python.org/mailman/listinfo/python-list
