On 30 Nov 2012, at 9:54 AM, Magnitus <eric_vallee2...@yahoo.ca> wrote:
> It is not executed, but given that models are not "compiled" into bytecode as
> far as I know, won't it have to be interpreted (incurring a performance
> penalty for the interpreter having to parse additional code)?
Ordinarily, in production, your files would be precompiled. I think you'd be
hard pressed to measure the additional overhead of a few extra lines in your
model.
>
> On Friday, 30 November 2012 12:50:33 UTC-5, Jonathan Lundell wrote:
> On 30 Nov 2012, at 9:38 AM, Magnitus <eric_va...@yahoo.ca> wrote:
>> What you are suggesting addresses the concurrency access problem (which
>> could be resolved with the DB since it locks or with your suggestion).
>>
>> However, if you stick the logic in your model, you still have the problem of
>> the logic appearing on every request.
>>
>> While you do speed up the checking with the "Test & set" instead of DB
>> access, eliminating the disk read, you do burden your model with additional
>> code bloat.
>>
>> I suppose that could be mitigated by inserting your script logic into a
>> module and passing the db as an argument to it.
>
> The overhead to check global_settings is negligibly more than an ordinary
> dict access, and except for the first request, that's all you'll incur. I'm
> pretty sure that's quite a bit less than an import of a module that's
> already imported (since it still has to check whether the module is already
> there).
>
> It's true that the db code will still be there, but it won't be executed.
>
>>
>> On Thursday, 29 November 2012 14:00:05 UTC-5, Jonathan Lundell wrote:
>> On 29 Nov 2012, at 10:22 AM, Magnitus <eric_va...@yahoo.ca> wrote:
>>> >There are a couple of issues. One is: what do you mean by startup? It's
>>> >fairly well defined if you're running web2py with Rocket as the server,
>>> >but in a wsgi environment, especially one that might have multiple
>>> >processes, it's a lot less clear.
>>>
>>> Yes and no.
>>>
>>> Obviously, a task that needs to run at startup would have to be robust,
>>> because you might start and stop your server multiple times (for example if
>>> it creates groups, it would have to check if the groups already exist
>>> before creating them).
>>>
>>> In that context, it isn't too much of a problem if it is executed multiple
>>> times if wsgi spawns multiple processes given that it is an overhead only
>>> when the server starts.
>>>
>>> However, if you absolutely need to make sure that the task itself runs only
>>> once even with multiple processes spawning, it seems like such
>>> synchronization could be handled in the DB (either as a feature of the
>>> framework or by the scripter himself, it's not too big of an overhead) or
>>> just by using another interprocess synchronisation mechanism that modern
>>> OSes provide.
>>>
>>> > So: what are you really trying to do? In what way is it a problem to run
>>> > on the first request after startup (whatever startup is) rather than at
>>> > startup itself? If you're initializing the database, can you check
>>> > that it's already initialized? (Yes, overhead, but would it work
>>> > logically?)
>>>
>>> I'm sure it's not THAT big of a performance overhead, but if possible, I
>>> prefer that tasks that only need to be executed at startup be only executed
>>> at startup.
>>>
>>> Not only does it run faster, but intuitively, I consider it to be better
>>> design as it modularize your code more since it doesn't mix up tasks that
>>> need to be executed at startup only with tasks that need to be executed
>>> with every request.
>>
>> How about something along these lines: set a flag in
>> gluon.settings.global_settings, using an app-unique key (because
>> global_settings should but doesn't have a per-application namespace you want
>> something like 'com.yourdomain.yourapp' or the like). Test & set the flag
>> early in your model, and do your db init conditional on the model not being
>> set, using db semantics to do it once, since you could have updates from
>> multiple processes. A straightforward mechanism (I think?) would be to just
>> do an insert with some appropriate field marked unique, and ignore
>> insert-collision exceptions.
>>
>>>
>>> On Thursday, 29 November 2012 10:44:17 UTC-5, Jonathan Lundell wrote:
>>> On 29 Nov 2012, at 6:41 AM, Magnitus <eric_va...@yahoo.ca> wrote:
>>>> I'd be curious to know if the lack of feedback on this thread so far is
>>>> due to the devs being incredibly busy or web2py lacking a genuine working
>>>> mechanism to execute scripts that have access to the models at startup
>>>> only.
>>>
>>> I've tried to do something like this myself (clear memcached on startup, in
>>> my case) and couldn't figure out a good way.
>>>
>>> There are a couple of issues. One is: what do you mean by startup? It's
>>> fairly well defined if you're running web2py with Rocket as the server, but
>>> in a wsgi environment, especially one that might have multiple processes,
>>> it's a lot less clear.
>>>
>>> Another is that the built-in cron mechanism was badly broken. ISTR seeing
>>> some fixes from Massimo go by recently, but I didn't look closely at what
>>> was going on.
>>>
>>> So: what are you really trying to do? In what way is it a problem to run on
>>> the first request after startup (whatever startup is) rather than at
>>> startup itself? If you're initializing the database, can you check that
>>> it's already initialized? (Yes, overhead, but would it work logically?)
>>>
>>>>
>>>> I've tried a lot of permutations for setting a script at the start with
>>>> cron and I'm now convinced that the functionality is either broken or so
>>>> obscure that most non-devs users that rely on the documentation alone
>>>> won't be able to make it work (at least, not without a major head ache).
>>>>
>>>> Besides, it seems from the documentation that cron is being phased out
>>>> anyways, so I'm thinking it would be nice to provide some sort of hook for
>>>> users to insert a script at startup with access to the models instead.
>>>>
>>>> Inserting "from gluon import *" in the module won't work either since the
>>>> db handles are not defined in gluon, but in the models.
>>>>
>>>> I'm guessing that maybe the entire database definition could be moved to a
>>>> module instead (I'd have to reflect on all the implications of this though
>>>> it would definitely increase the performance at the cost of having to
>>>> reboot when you want to modify the db's structure), but that's not the
>>>> default out of the box architecture.
>>>>
>>>> Either way, inserting the script into a module that is included in a
>>>> controller is not a 100% satisfying solution since the script is actually
>>>> executed on the first page request and not really at startup.
>>>>
>>>> On Wednesday, 28 November 2012 12:55:00 UTC-5, Magnitus wrote:
>>>> After some reflection, I suppose I could write a wrapper script that first
>>>> runs python web2py.py --import_models --shell=<MyController>/default
>>>> --run="<My Script>" and then starts web2py normally when using web2py
>>>> standalone.
>>>>
>>>> That is one workaround.
>>>>
>>>> On Wednesday, 28 November 2012 11:19:01 UTC-5, Magnitus wrote:
>>>> Hi gents,
>>>>
>>>> I have been populating my databases with groups (for access control)
>>>> manually for a while and then, switched to running a script that does it,
>>>> though I still run the script manually in the web2py context.
>>>>
>>>> Now, I'd like to programmatically implement this in a sensible way (in a
>>>> way that runs by itself and doesn't need human intervention at all).
>>>>
>>>> Obviously, you could just put the logic that does this in a model, but I
>>>> don't like the idea of needlessly increasing the size of my models with
>>>> this or running the logic to check the existance of and generate groups
>>>> for each request.
>>>>
>>>> Another alternative would be to put the logic in a module that would be
>>>> imported (with the idea that it would run on the first import which is
>>>> done only once), but then imported modules don't have access to the web2py
>>>> objects (notably, my database handle) unless you pass them on as
>>>> parameters which you cannot do in a module's global scope when it is first
>>>> included.
>>>>
>>>> This leaves me with cron and running it as a cron job at boot time.
>>>> However, it isn't working.
>>>>
>>>> I tried running my script using the web2py context (ie, python web2py.py
>>>> --import_models --shell=<MyController>/default --run="<My Script>") and it
>>>> ran fine and generated the db entries I wanted.
>>>>
>>>> Then, I started my web2py execution with cron activated (ie, python
>>>> web2py.py -Y).
>>>>
>>>> I tried the two varations portayed in the example for the crontab
>>>> ("@reboot * * * * root *applications/<My App>/cron/Generate_groups.py" and
>>>> the shorter "@reboot root *applications/<My App>/cron/Generate_groups.py).
>>>>
>>>> And yes, my db transactions end with a db.commit().
>>>>
>>>> I've been banging my head against this for two hours and while I'm sure
>>>> whatever is wrong with this will come to me over time, this is definitely
>>>> a case of sooner would be better than later.
>>>>
>>>> Insights would be appreciated.
>>>>
>>>> --
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>>
>>>
>>>
>>
>>
>>
>> --
>>
>>
>>
>
>
>
> --
>
>
>
--
