Web framework to recommend
Hi all, I just started learning Python and would like to starting writing some web-based applications with Python. I did have pretty much experience with doing so with PHP and Java, but Python seems a bit different for me. Do you guys have some good web framework to recommend? (I don't want to use CGI mode) Thanks a lot. -- Jacky -- http://mail.python.org/mailman/listinfo/python-list
Re: Web framework to recommend
Hi all, Thanks all of you for your recommendations, I'll check them out one by one. Besides, I'd like to say more about what I want to do. Since I'm planning to write something for "backend", I might not focus very much on the layout (HTML template) yet. Instead, I'd like to receive inputs in XML format and then output XML after processing (or maybe I should use web services, perhaps?). Of course, I will write a complete web application in my other projects (such as those I wrote with PRADO and JSF), so all recommendations are welcomed :) Thank you! Best Wishes, Jacky Jacky wrote: > Hi all, > > I just started learning Python and would like to starting writing some > web-based applications with Python. > > I did have pretty much experience with doing so with PHP and Java, but > Python seems a bit different for me. > > Do you guys have some good web framework to recommend? (I don't want to > use CGI mode) > > Thanks a lot. > > -- > Jacky -- http://mail.python.org/mailman/listinfo/python-list
Discount (paypal) www.ebuyvip.com nfl jersey ugg boots
Hi friend Thanks for your reply. The current stock list with the available jerseys and sizea in the add annex, pls check it. and the jerseys pictures you can check my ablum : http://likenfl.photo.163.com . the moq order: 10 pcs the price : $20 inc shipping fee and tax.If you worry do businees with me first. and cann't trust me, the payment we can by paypal. that we all can trust other. If you have any questions pls tell me , i will give you a satisfaction answer. Hope we can have a nice and long term cooperation. UGG BOOTS: $45 NFL JERSEY: $20 Website: www.ebuyvip.com MSN: ebuy...@hotmail.com Waiting for your good news. Best Regard Jarky -- http://mail.python.org/mailman/listinfo/python-list
How to convert bytearray into integer?
Hi there,
Recently I'm facing a problem to convert 4 bytes on an bytearray into
an 32-bit integer. So far as I can see, there're 3 ways: a) using
struct module, b) using ctypes module, and c) manually manipulation.
Are there any other ways?
My sample is as following:
-
import struct
import ctypes
def test_struct(buf, offset):
return struct.unpack_from("I", buf, offset)[0]
def test_ctypes(buf, offset):
return ctypes.c_uint32.from_buffer(buf, offset).value
def test_multi(buf, offset):
return buf[offset] + (buf[offset+1] << 8) + (buf[offset+2] << 16) +
(buf[offset+3] << 24)
buf_w = bytearray(5)
buf_w[1] = 1
buf_r = buffer(buf_w)
if __name__ == '__main__':
import timeit
t1 = timeit.Timer("test_struct(buf_r, 1)",
"from __main__ import test_struct, buf_r")
t2 = timeit.Timer("test_ctypes(buf_w, 1)",
"from __main__ import test_ctypes, buf_w")
t3 = timeit.Timer("test_multi(buf_w, 1)",
"from __main__ import test_multi, buf_w")
print t1.timeit(number=1000)
print t2.timeit(number=1000)
print t3.timeit(number=1000)
-
Yet the results are bit confusing:
-
number = 1
0.0081958770752
0.012549161911
0.0112121105194
number = 1000
0.00087308883667
0.00125789642334
0.00110197067261
number = 100
0.917911529541 # 9.17911529541e-05
0.000133991241455
0.00011420249939
number = 10
1.69277191162e-05
2.19345092773e-05
1.69277191162e-05
number = 1
1.00135803223e-05
1.00135803223e-05
5.96046447754e-06
-
As the number of benchmarking loops decreasing, method c which is
manually manipulating overwhelms the former 2 methods. However, if
number == 10K, the struct method wins.
Why does it happen?
Thanks,
Jacky (jacky.chao.wang#gmail.com)
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to convert bytearray into integer?
Hi Thomas, Thanks for your comments! Please check mine inline. On Aug 17, 1:50 am, Thomas Jollans wrote: > On Monday 16 August 2010, it occurred to Jacky to exclaim: > > > Hi there, > > > Recently I'm facing a problem to convert 4 bytes on an bytearray into > > an 32-bit integer. So far as I can see, there're 3 ways: > > a) using struct module, > > Yes, that's what it's for, and that's what you should be using. My concern is that struct may need to parse the format string, construct the list, and de-reference index=0 for this generated list to get the int out. There should be some way more efficient? > > > b) using ctypes module, and > > Yeeaah, that would work, but that's really not what it's for. from_buffer > wants a writable buffer interface, which is unlikely to be what you want. Actually my buffer is writable --- it's an bytearray. Turning it into a R/O one make me to do extra effort: wrapping the bytearray into buffer(). My question is, this operation seems like to be much simpler than the former one, and it's very straightforward as well. Why is it slow? > > > c) manually manipulation. > > Well, yes, you can do that, but it gets messy when you're working with more > complex data structures, or you have to consider byte order. agree. :) > > > Are there any other ways? > > You could write a C extension module tailored to your specific purpose ;-) Ha, yes. Actually I've already modified socketmodule.c myself --- it's hard to image why socket object provides the interface: socket.recv_from(buf[, num_bytes[, flags]]) but forget the more generic one: socket.recv_from(buf[, offset[, num_bytes[, flags]]]) So do socket.send(...). > > > number = 1 > > 1.00135803223e-05 > > 1.00135803223e-05 > > 5.96046447754e-06 > > - > > > As the number of benchmarking loops decreasing, method c which is > > manually manipulating overwhelms the former 2 methods. However, if > > number == 10K, the struct method wins. > > > Why does it happen? > > struct wins because it's built for the job. > > As for the small numbers: don't take these numbers seriously. Just don't. This > may be caused by the way your OS's scheduler handles things for all I know. If > there is an explanation for this unscientific observation, I have two guesses > what it might be: > * struct and ctypes still need to do some setup work, or something > * somebody is optimising something, but doesn't know what they should be > optimising in the first place after only a few iterations. Agree. Thanks. - Jacky -- http://mail.python.org/mailman/listinfo/python-list
Re: How to convert bytearray into integer?
Hi Mark,
Thanks for your reply. Agree and I'll use your suggestions. Thanks!
-Jacky
On Aug 17, 3:36 am, Mark Dickinson wrote:
> On Aug 16, 8:08 pm, Jacky wrote:
>
> > Hi Thomas,
>
> > Thanks for your comments! Please check mine inline.
>
> > On Aug 17, 1:50 am, Thomas Jollans wrote:
>
> > > On Monday 16 August 2010, it occurred to Jacky to exclaim:
>
> > > > Hi there,
>
> > > > Recently I'm facing a problem to convert 4 bytes on an bytearray into
> > > > an 32-bit integer. So far as I can see, there're 3 ways:
> > > > a) using struct module,
>
> > > Yes, that's what it's for, and that's what you should be using.
>
> > My concern is that struct may need to parse the format string,
> > construct the list, and de-reference index=0 for this generated list
> > to get the int out.
>
> > There should be some way more efficient?
>
> Well, you can improve on the struct solution by using the
> struct.Struct class to avoid parsing the format string repeatedly:
>
> >>> import struct
> >>> S = struct.Struct(' >>> S.unpack_from(buffer(bytearray([1,2,3,4,5])))
>
> (67305985,)
>
> This doesn't make a huge difference on my machine (OS X 10.6.4, 64-bit
> build of Python 2.6) though; it's probably more effective for long
> format strings. Adding:
>
> def test_struct2(buf, offset, S=struct.Struct(' return S.unpack_from(buf, offset)[0]
>
> to your test code, I see a speedup of around 8% over your test_struct.
>
> By the way, you may want to consider using an explicit byte-order/size
> marker in your format string; i.e., use ' forces a 4-byte little-endian interpretation, regardless of the
> platform you're running Python on.
>
> --
> Mark- Hide quoted text -
>
> - Show quoted text -
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to convert bytearray into integer?
On Aug 17, 3:38 am, Thomas Jollans wrote: > On Monday 16 August 2010, it occurred to Jacky to exclaim: > > > > > > > Hi Thomas, > > > Thanks for your comments! Please check mine inline. > > > On Aug 17, 1:50 am, Thomas Jollans wrote: > > > On Monday 16 August 2010, it occurred to Jacky to exclaim: > > > > Hi there, > > > > > Recently I'm facing a problem to convert 4 bytes on an bytearray into > > > > an 32-bit integer. So far as I can see, there're 3 ways: > > > > a) using struct module, > > > > Yes, that's what it's for, and that's what you should be using. > > > My concern is that struct may need to parse the format string, > > construct the list, and de-reference index=0 for this generated list > > to get the int out. > > > There should be some way more efficient? > > The struct module is written in C, not in Python. It does have to parse a > string, yes, so, if you wrote your own, limited, C function to do the job, it > might be marginally faster. > > > > > > > b) using ctypes module, and > > > > Yeeaah, that would work, but that's really not what it's for. from_buffer > > > wants a writable buffer interface, which is unlikely to be what you want. > > > Actually my buffer is writable --- it's an bytearray. Turning it into > > a R/O one make me to do extra effort: wrapping the bytearray into > > buffer(). > > > My question is, this operation seems like to be much simpler than the > > former one, and it's very straightforward as well. Why is it slow? > > Unlike struct, it constructs an object you're not actually interested in > around your int. > > > it's hard to image why socket object provides the interface: > > socket.recv_from(buf[, num_bytes[, flags]]) but forget the more > > generic one: socket.recv_from(buf[, offset[, num_bytes[, flags]]]) > > Well, that's what pointer arithmetic (in C) or slices (in Python) are for! > There's an argument to be made for sticking close to the traditional > (originally C) interface here - it's familiar. Hi Thomas, - I'm not quite follow you. It will be great if you could show me some code no this part... > > - Thomas- Hide quoted text - > > - Show quoted text - -- http://mail.python.org/mailman/listinfo/python-list
Re: How to convert bytearray into integer?
On Aug 17, 3:53 am, Mark Dickinson wrote:
> On Aug 16, 8:36 pm, Mark Dickinson wrote:
>
>
>
>
>
> > On Aug 16, 8:08 pm, Jacky wrote:
> > > My concern is that struct may need to parse the format string,
> > > construct the list, and de-reference index=0 for this generated list
> > > to get the int out.
>
> > > There should be some way more efficient?
>
> > Well, you can improve on the struct solution by using the
> > struct.Struct class to avoid parsing the format string repeatedly:
>
> > >>> import struct
> > >>> S = struct.Struct(' > >>> S.unpack_from(buffer(bytearray([1,2,3,4,5])))
>
> > (67305985,)
>
> > This doesn't make a huge difference on my machine (OS X 10.6.4, 64-bit
> > build of Python 2.6) though; it's probably more effective for long
> > format strings.
>
> Sorry, this was inaccurate: this makes almost *no* significant
> difference on my machine for large test runs (1 and up). For
> small ones, though, it's faster. The reason is that the struct module
> caches (up to 100, in the current implementation) previously used
> format strings, so with your tests you're only ever parsing the format
> string once anyway. Internally, the struct module converts that
> format string to a Struct object, and squirrels that Struct object
> away into its cache, which is implemented as a dict from format
> strings to Struct objects. So the next time that the format string is
> used it's simply looked up in the cache, and the Struct object
> retrieved.
>
> By the way, in Python 3.2 there's yet another fun way to do this,
> using int.from_bytes.
>
> >>> int.from_bytes(bytearray([1,2,3,4]), 'little')
Thanks! It looks pretty like the ctypes way. ;)
>
> 67305985
>
> --
> Mark- Hide quoted text -
>
> - Show quoted text -
--
http://mail.python.org/mailman/listinfo/python-list
Re: Deadlock problem using multiprocessing
> > I get this exception when I run the first program: > > Exception in thread Thread-1: > Traceback (most recent call last): > File "/usr/lib/python3.1/threading.py", line 516, in _bootstrap_inner > self.run() > File "/usr/lib/python3.1/threading.py", line 469, in run > self._target(*self._args, **self._kwargs) > File "/usr/lib/python3.1/multiprocessing/pool.py", line 231, in > _handle_tasks > put(task) > File "/usr/lib/python3.1/pickle.py", line 1349, in dumps > Pickler(f, protocol, fix_imports=fix_imports).dump(obj) > _pickle.PicklingError: Can't pickle : attribute lookup > builtins.method failed > > There is no deadlock. You are hitting this > problem:http://stackoverflow.com/questions/1816958/cant-pickle-type-instancem... > > -- > regards, > kushal You're right. I just realized that I was running the program through my Vim plug-in which would use the subprocess module to open the process, wait for its termination and get the output. It seems that multi-process program would act quite differently such that my Vim plug-in had been halted without getting any error information, so I was to falsely take it as a deadlock. The thread in stackoverflow you referenced is great, now I'm trying to fit the solution in my own program, thanks a lot! -- http://mail.python.org/mailman/listinfo/python-list
how to debug python application crashed occasionally
Hello recently, I met a problem with one python application running with python2.5 | debian/lenny adm64 system: it crashed occasionally in our production environment. The problem started to happen just after we upgraded the python application from python2.4 | debian/etch amd64. after configuring the system to enable core dump & debugging with the core dumps by following the guide line from http://wiki.python.org/moin/DebuggingWithGdb, I became more confused about that. The first crash case was happening in calling python-xml module, which is claimed as a pure python module, and it's not supposed to crash python interpreter. because the python application is relatively a big one, I can not show u guys the exact source code related with the crash, but only the piece of python modules. GDB shows it's crashed at string join operation: #0 string_join (self=0x7f7075baf030, orig=) at ../Objects/stringobject.c:1795 1795../Objects/stringobject.c: No such file or directory. in ../Objects/stringobject.c and pystack macro shows the details: gdb) pystack /usr/lib/python2.5/StringIO.py (271): getvalue /usr/lib/python2.5/site-packages/_xmlplus/dom/minidom.py (62): toprettyxml /usr/lib/python2.5/site-packages/_xmlplus/dom/minidom.py (47): toxml at that time, we also found python-xml module has performance issue for our application, so we decided to use python-lxml to replace python-xml. After that replacement, the crash was gone. That's a bit weird for me, but anyway, it's gone. Unfortunately, another two 'kinds' of crashes happening after that, and the core dumps show they are not related with the replacement. One is crashed with "Program terminated with signal 11", and the pystack macro shows it's crashed at calling the built-in id() function. #0 visit_decref (op=0x20200a3e22726574, data=0x0) at ../Modules/ gcmodule.c:270 270 ../Modules/gcmodule.c: No such file or directory. in ../Modules/gcmodule.c Another is crashed with "Program terminated with signal 7", and the pystack macro shows it's crashed at the exactly same operation (string join) as the first one (python-xml), but in different library python- simplejson: #0 string_join (self=0x7f5149877030, orig=) at ../Objects/stringobject.c:1795 1795../Objects/stringobject.c: No such file or directory. in ../Objects/stringobject.c (gdb) pystack /var/lib/python-support/python2.5/simplejson/encoder.py (367): encode /var/lib/python-support/python2.5/simplejson/__init__.py (243): dumps I'm not good at using gdb & C programming, then I tried some other ways to dig further: * get the source code of python2.5, but can not figure out the crash reason :( * since Debian distribution provides python-dbg package, and I tried to use python2.5-dbg interpreter, but not the python2.5, so that I can get more debug information in the core dump file. Unfortunately, my python application is using a bunch of C modules, and not all of them provides -dbg package in Debian/Lenny. So it still doesn't make any progress yet. I will be really appreciated if somebody can help me about how to debug the python crashes. Thanks in advance! BR Jacky Wang -- http://mail.python.org/mailman/listinfo/python-list
Re: how to debug python application crashed occasionally
could anyone help me? On Apr 21, 2:55 pm, jacky wang wrote: > Hello > > recently, I met a problem with one python application running with > python2.5 | debian/lenny adm64 system: it crashed occasionally in our > production environment. The problem started to happen just after we > upgraded the python application from python2.4 | debian/etch amd64. > > after configuring the system to enable core dump & debugging with > the core dumps by following the guide line > fromhttp://wiki.python.org/moin/DebuggingWithGdb, > I became more confused about that. > > The first crash case was happening in calling python-xml module, > which is claimed as a pure python module, and it's not supposed to > crash python interpreter. because the python application is relatively > a big one, I can not show u guys the exact source code related with > the crash, but only the piece of python modules. GDB shows it's > crashed at string join operation: > > #0 string_join (self=0x7f7075baf030, orig=) > at ../Objects/stringobject.c:1795 > 1795 ../Objects/stringobject.c: No such file or directory. > in ../Objects/stringobject.c > > and pystack macro shows the details: > > gdb) pystack > /usr/lib/python2.5/StringIO.py (271): getvalue > /usr/lib/python2.5/site-packages/_xmlplus/dom/minidom.py (62): > toprettyxml > /usr/lib/python2.5/site-packages/_xmlplus/dom/minidom.py (47): toxml > > at that time, we also found python-xml module has performance issue > for our application, so we decided to use python-lxml to replace > python-xml. After that replacement, the crash was gone. That's a bit > weird for me, but anyway, it's gone. > > Unfortunately, another two 'kinds' of crashes happening after that, > and the core dumps show they are not related with the replacement. > > One is crashed with "Program terminated with signal 11", and the > pystack macro shows it's crashed at calling the built-in id() > function. > > #0 visit_decref (op=0x20200a3e22726574, data=0x0) at ../Modules/ > gcmodule.c:270 > 270 ../Modules/gcmodule.c: No such file or directory. > in ../Modules/gcmodule.c > > Another is crashed with "Program terminated with signal 7", and the > pystack macro shows it's crashed at the exactly same operation (string > join) as the first one (python-xml), but in different library python- > simplejson: > > #0 string_join (self=0x7f5149877030, orig=) > at ../Objects/stringobject.c:1795 > 1795 ../Objects/stringobject.c: No such file or directory. > in ../Objects/stringobject.c > > (gdb) pystack > /var/lib/python-support/python2.5/simplejson/encoder.py (367): encode > /var/lib/python-support/python2.5/simplejson/__init__.py (243): dumps > > I'm not good at using gdb & C programming, then I tried some other > ways to dig further: > * get the source code of python2.5, but can not figure out the > crash reason :( > * since Debian distribution provides python-dbg package, and I > tried to use python2.5-dbg interpreter, but not the python2.5, so that > I can get more debug information in the core dump file. Unfortunately, > my python application is using a bunch of C modules, and not all of > them provides -dbg package in Debian/Lenny. So it still doesn't make > any progress yet. > > I will be really appreciated if somebody can help me about how to > debug the python crashes. > > Thanks in advance! > > BR > Jacky Wang -- http://mail.python.org/mailman/listinfo/python-list
