Hi AchipA
I think most of the work is done already with regard to how to set up
the environment and do an exec without Popen. If you take a look at
the run function in gluon\shell.py it has the signature below.
I don't see why the run function cannot be copied and edited with a
small number of changes to allow for a Python internal exec (without a
shell) of an arbitrary python file or string, with an expected
environment.
I agree there is more potential for users to shoot themselves in the
foot with cron jobs! The door has been opened. Why stop now? The world
is still revolving, the sun is still shining and the sky has not
fallen down because of users misbehaving!
John Heenan
def run(
appname,
plain=False,
import_models=False,
startfile=None,
):
"""
Start interactive shell or run Python script (startfile) in web2py
controller environment. appname is formatted like:
a web2py application name
a/c exec the controller c into the application environment
"""
On Feb 9, 12:34 pm, AchipA <attila.cs...@gmail.com> wrote:
> John, I did not mean fight about principles or dismiss your
> suggestion.
> I perfectly understand what you need and see that rationale in it.
> It's
> just that if I'm doing an overhaul, I'd like to do it right - leaving
> to the users to 'behave' and not shoot themselves in the foot is
> something I'd like to avoid. Exec is not as easy as it sounds as cron
> operates really really low, in a separate thread on the level just
> above
> WSGI/webserver. I have access to very little of web2py's features
> 'down
> there' so things get tricky real fast (especially if you want to
> include
> models in the story) - that's why calling a new web2py process to sort
> or controller/model issues was a simple way of doing it in a
> foolproof manner.
>
> On Feb 9, 3:14 am, John Heenan <johnmhee...@gmail.com> wrote:
>
> > Since AchipA is working on other changes to cron.py and cron.py is
> > his, I think I should wait to see if AchipA includes additons with
> > Massimo's preferred format below before being so bold as to submit a
> > patch myself.
>
> > John Heenan
>
> > On Feb 9, 10:54 am, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > > I have no objection to adding this feature. I do not like the !*"
> > > prefix.
>
> > > I think we can do:
>
> > > @hourly root file -> exec file in Popen
> > > @hourly root file.py -> exec file in Popen with python
> > > @hourly root *file.py -> exec file in Popen with python -S <app> -M -R
> > > <file>
> > > @hourly root **file.py -> exec file in Popen with python -S <app> -R
> > > <file>
> > > @hourly root *file -> exec file in Popen with python -S <file> -M
> > > @hourly root **file -> exec file in Popen with python -S <file>
> > > @hourly root *"data" -> exec "data" without Popen with environment as -
> > > M
> > > @hourly root **"data" -> exec "data" without Popen without environment
> > > as -M
> > > @hourly root *"filename" -> execfile filename without Popen with
> > > environment as -M
> > > @hourly root **"filename" -> execfile filename without Popen without
> > > environment as -M
> > > (we can distinguish data from filename checking if "..." starts with
> > > './' or '/'.)
>
> > > I would take a patch in this direction. You only add the last 4
> > > options at the end of the cron file. There is nothing else that needs
> > > to be done I think.
>
> > > Massimo
>
> > > All is needed it a
>
> > > On Feb 8, 6:30 pm, John Heenan <johnmhee...@gmail.com> wrote:
>
> > > > Hi
>
> > > > I stated "My point is can we have an option to:"
>
> > > > Maybe I should code up a patch for cron.py and submit it for
> > > > examination. Cron.py is impressive, I am not criticising it or
> > > > attacking its approach.
>
> > > > The option I am requesting are ADDITIONS to cron.py that are TRIVIAL
> > > > to code in as additions to cron.py. Web2py already has a prepared
> > > > environment ready to be passed for exec use.
>
> > > > To reiterate, the option I am requesting is an additional option (not
> > > > a replacement) to do an exec (or execfile) on a file that ends
> > > > with .py (or .pyc) AND an option to do an exec on a string (such as
> > > > "import a; a.mycronaction()")
>
> > > > For example, suppose ! is chosen to indicate not to use the default of
> > > > the Popen class wrapper but to use exec instead, then the following is
> > > > an example of use:
>
> > > > */10 * * * * root !
> > > > **expire_sessions.py #use the web2py environment with execfile on
> > > > the .py file but do not execute models
> > > > or
> > > > */10 * * * * root !*"import a;
> > > > a.mycronaction()" #use the web2py environment with exec on the string
> > > > and include execution of models
>
> > > > I do not want to become involved in a grand philosophical or
> > > > ideological debate about the merits or otherwise of using the Popen
> > > > wrapper class verses the internal Python exec (or execfile) function.
> > > > For trivial tasks (such as deleting stale session files) the Popen
> > > > class wrapper is overkill. Using Popen involves unnecessary extra RAM
> > > > use and process invocation. Invoking a Python process is very
> > > > expensive in CPU and RAM terms. In fact Python is impractical in a web
> > > > environment unless the Python process remains persistent, This is the
> > > > reason why hosting is relatively expensive for python web solutions
> > > > compared to PHP and Perl.
>
> > > > While such concerns might seem trivial for tasks that are not repeated
> > > > often, it is not trivial if hosting is in an environment where there
> > > > are penalties for exceeding allocated RAM, such as I believe occurs at
> > > > Webfaction, popular with those trialling web2py.
>
> > > > For trivial housekeeping tasks, just a minimal (non models)
> > > > environment associated with the application is adequate, such as is
> > > > currently passed in for use with Popen.
>
> > > > If tasks are so involved that processes are at a non negligible risk
> > > > of choking then users will hardly be concerned about using execfile
> > > > instead of Popen and will instead code up an independent supervisory
> > > > process outside web2py.
>
> > > > Deadlock or race conditions are not any more of an issue using exec
> > > > than with normal code (provided the code passed in for exec does
> > > > nothing out of the ordinary). A new thread is not invoked.
>
> > > > As is, cron.py waits for a Popen invoked process to end without a
> > > > supervisor which can result in a lock up of web2py.
>
> > > > It can be argued that there is a lack of hard quantified information
> > > > about the resources used by Python based web frameworks. Those who
> > > > develop websites and GUIs have frequently little care about boring
> > > > issues of resource usage. I bothered to take a look and was horrified
> > > > by the memory usage of Python. We need to show respect for practical
> > > > issues associated with using Python. Otherwise web2py and other
> > > > frameworks risk becoming relegated to academic toys that are from
> > > > 'enterprise' class. Java based frameworks are fighting back. Sun is
> > > > pushing GlassFish heavily into the enterprise environment.
>
> > > > Python based frameworks cannot expect to be competitive with PHP at
> > > > what PHP is best at (low resource usage) if Python frameworks don't
> > > > address issues in a practical way that are holding back their adoption
> > > > (such as resource usage leading to expensive hosting).
>
> > > > John Heenan
>
> > > > On Feb 8, 11:40 pm, AchipA <attila.cs...@gmail.com> wrote:
>
> > > > > Fiddling with this now, but have a few concerns, so I'd like Massimo
> > > > > to chime is as the exec expert. The main reason for going POpen is to
> > > > > have a clean, thread safe environment, and with exec I'm not sure we
> > > > > won't hit race conditions or deadlock possibilities, especially if we
> > > > > touch on models. Some help how to exec stuff with the proper
> > > > > environment would be welcome :)
>
> > > > > Another thing to note is that when execing, we no longer have control
> > > > > over what we execute - i.e. while we can kill/clean zombie POpened
> > > > > processes, if we get stuck in just another thread (=with exec) it's
> > > > > stuck there until the whole web2py/webserver is restarted.
>
> > > > > On Jan 31, 7:27 am, John Heenan <johnmhee...@gmail.com> wrote:
>
> > > > > > I have not get my point across.
>
> > > > > > By 'Python exec' I mean using the 'exec' internal Python command. I
> > > > > > don't mean 'Python exec' as a shorthand for 'execute a new Unix,
> > > > > > Windows or Mac OS process under the ownership of the cron code
> > > > > > written
> > > > > > in Python'.
>
> > > > > > I have just examined the most recent trunk version of cron.py
> > > > > > inhttp://code.google.com/p/web2py/source/browse/gluon/contrib/cron.py
>
> > > > > > For .py files the Popen command is still executed using the
> > > > > > cronlauncher class that uses Popen
>
> > > > > > For those of us trying ot keep RAM usage down, who don't want to
> > > > > > use a
> > > > > > regular crontab and who only want to run Python files, using Popen
> > > > > > is
> > > > > > not necessasry.
>
> > > > > > My point is can we have an option to:
>
> > > > > > 1) Just do an Python exec on an opened file contents (with ot
> > > > > > without
> > > > > > the models envrionment), instead of doing a Popen
> > > > > > 2) Have an option to do an Python exec on a string, such as on
> > > > > > "import
> > > > > > a; a.mycronaction()"
>
> > > > > > Thanks
>
> > > > > > John Heenan
>
> > > > > > On Jan 31, 2:17 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > > > > > > Unless we misunderstand we have that already. The file just needs
> > > > > > > to
> > > > > > > end in .py as in
>
> > > > > > > * * * * * root myscript.py
>
> > > > > > > You can also call actions
>
> > > > > > > * * * * * root *myscript.py
>
> > > > > > > so that models are imported.
>
> > > > > > > On Jan 30, 8:36 pm, John Heenan <johnmhee...@gmail.com> wrote:
>
> > > > > > > > Hi AchipA
>
> > > > > > > > Another feature request, if the cron file is a python file, how
> > > > > > > > about
> > > > > > > > a cron option to open the file and do an exec on the file
> > > > > > > > contents?
>
> > > > > > > > Thanks
>
> > > > > > > > John Heenan
>
> > > > > > > > On Jan 31, 12:33 pm, John Heenan <johnmhee...@gmail.com> wrote:
>
> > > > > > > > > Hi AchipA
>
> > > > > > > > > Your cron is nice work and neatly deals with the messy side of
> > > > > > > > > launching an independent OS process and waiting for it to
> > > > > > > > > finish.
>
> > > > > > > > > For those of us who do not wish to launch an independent
> > > > > > > > > process (to
> > > > > > > > > keep down memory use) and are happy to stick with Python for
> > > > > > > > > cron
> > > > > > > > > jobs, following is a request to add in the following feature
> > > > > > > > > to your
> > > > > > > > > cron fpr web2py.
>
> > > > > > > > > Provide a mechanism to just perform a Python exec on a string
> > > > > > > > > statment.
>
> > > > > > > > > For example a string statement might be
> > > > > > > > > "import a; a.mycronaction()"
>
> > > > > > > > > Thanks
>
> > > > > > > > > John Heenan
>
> > > > > > > > > On Jan 31, 6:52 am, AchipA <attila.cs...@gmail.com> wrote:
>
> > > > > > > > > > Just to chime in, I'm still alive and preparing an update
> > > > > > > > > > to cron as
> > > > > > > > > > quite
>
--
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To post to this group, send email to web...@googlegroups.com.
To unsubscribe from this group, send email to
web2py+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/web2py?hl=en.
