Jayakumar, Are you sure that the code does exactly the same thing on PHP and Python? Send the code here if you can. It could depend on what you do with the grep output or how many times you spawn grep.
mic 2013/10/4 Ricardo Pedroso <rmdpedr...@gmail.com> > On Fri, Oct 4, 2013 at 8:33 AM, Jayakumar Bellie <jkumar.o...@gmail.com>wrote: > >> >> I try to run a grep command on a 30 MB file and redirect output to a tmp >> folder. >> Read the file in the tmp folder and display it. >> >> When I do this in PHP it is very fast. >> But when I do it in web2py it is slow. >> >> > This is fast for me with a sample csv of 60MB taken from > http://www.briandunning.com/sample-data/350000.zip > and changed the line endings from mac to unix. > > # coding: utf-8 > import sys > import os > from subprocess import call > > def index(): > response.headers['Content-Type'] = 'text/plain' > TEMPFOLDER = os.path.join(request.folder, 'private') > SOURCEFILE = os.path.join(TEMPFOLDER, '350000.csv') > TEMPFILE = os.path.join(TEMPFOLDER, 'tempfile.tmp') > > print >>sys.stderr, 'grep file' > try: > retcode = call('grep "A" %s >%s' % (SOURCEFILE, TEMPFILE), > shell=True) > if retcode < 0: > raise HTTP(404, "Child was terminated by signal %s" % > str(-retcode)) > elif retcode > 0: > raise HTTP(404, "Child returned %s" % str(retcode)) > except OSError as e: > raise HTTP(500, "Execution failed: %r" % e) > > print >>sys.stderr, 'read file' > return response.stream(open(TEMPFILE, 'rb'), chunk_size=10**6) > > > The temp file in this example will have 40MB > > Tested with: > > $ curl -o /dev/null -s -w "time for first byte: > %{time_starttransfer}s\ntotal time: %{time_total}s\n" > http://localhost:8000/bigfile > time for first byte: 1.203s > total time: 1.384s > > Most time is spent on grep. > > > Ricardo > > -- > Resources: > - http://web2py.com > - http://web2py.com/book (Documentation) > - http://github.com/web2py/web2py (Source code) > - https://code.google.com/p/web2py/issues/list (Report Issues) > --- > You received this message because you are subscribed to the Google Groups > "web2py-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to web2py+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
