It is not executed, but given that models are not "compiled" into bytecode
as far as I know, won't it have to be interpreted (incurring a performance
penalty for the interpreter having to parse additional code)?
On Friday, 30 November 2012 12:50:33 UTC-5, Jonathan Lundell wrote:
>
> On 30 Nov 2012, at 9:38 AM, Magnitus <eric_va...@yahoo.ca <javascript:>>
> wrote:
>
> What you are suggesting addresses the concurrency access problem (which
> could be resolved with the DB since it locks or with your suggestion).
>
> However, if you stick the logic in your model, you still have the problem
> of the logic appearing on every request.
>
> While you do speed up the checking with the "Test & set" instead of DB
> access, eliminating the disk read, you do burden your model with additional
> code bloat.
>
> I suppose that could be mitigated by inserting your script logic into a
> module and passing the db as an argument to it.
>
>
> The overhead to check global_settings is negligibly more than an ordinary
> dict access, and except for the first request, that's all you'll incur. I'm
> pretty sure that's quite a bit less than an import of a module that's
> already imported (since it still has to check whether the module is already
> there).
>
> It's true that the db code will still be there, but it won't be executed.
>
>
> On Thursday, 29 November 2012 14:00:05 UTC-5, Jonathan Lundell wrote:
>>
>> On 29 Nov 2012, at 10:22 AM, Magnitus <eric_va...@yahoo.ca> wrote:
>>
>> >There are a couple of issues. One is: what do you mean by startup? It's
>> fairly well defined if you're running web2py with Rocket as the server, but
>> in a wsgi environment, especially one that might have multiple processes,
>> it's a lot less clear.
>>
>> Yes and no.
>>
>> Obviously, a task that needs to run at startup would have to be robust,
>> because you might start and stop your server multiple times (for example if
>> it creates groups, it would have to check if the groups already exist
>> before creating them).
>>
>> In that context, it isn't too much of a problem if it is executed
>> multiple times if wsgi spawns multiple processes given that it is an
>> overhead only when the server starts.
>>
>> However, if you absolutely need to make sure that the task itself runs
>> only once even with multiple processes spawning, it seems like such
>> synchronization could be handled in the DB (either as a feature of the
>> framework or by the scripter himself, it's not too big of an overhead) or
>> just by using another interprocess synchronisation mechanism that modern
>> OSes provide.
>>
>> > So: what are you really trying to do? In what way is it a problem to
>> run on the first request after startup (whatever startup is) rather than at
>> startup itself? If you're initializing the database, can you check that
>> it's already initialized? (Yes, overhead, but would it work logically?)
>>
>> I'm sure it's not THAT big of a performance overhead, but if possible, I
>> prefer that tasks that only need to be executed at startup be only executed
>> at startup.
>>
>> Not only does it run faster, but intuitively, I consider it to be better
>> design as it modularize your code more since it doesn't mix up tasks that
>> need to be executed at startup only with tasks that need to be executed
>> with every request.
>>
>>
>> How about something along these lines: set a flag in
>> gluon.settings.global_settings, using an app-unique key (because
>> global_settings should but doesn't have a per-application namespace you
>> want something like 'com.yourdomain.yourapp' or the like). Test & set the
>> flag early in your model, and do your db init conditional on the model not
>> being set, using db semantics to do it once, since you could have updates
>> from multiple processes. A straightforward mechanism (I think?) would be to
>> just do an insert with some appropriate field marked unique, and ignore
>> insert-collision exceptions.
>>
>>
>> On Thursday, 29 November 2012 10:44:17 UTC-5, Jonathan Lundell wrote:
>>>
>>> On 29 Nov 2012, at 6:41 AM, Magnitus <eric_va...@yahoo.ca> wrote:
>>>
>>> I'd be curious to know if the lack of feedback on this thread so far is
>>> due to the devs being incredibly busy or web2py lacking a genuine working
>>> mechanism to execute scripts that have access to the models at startup only.
>>>
>>>
>>> I've tried to do something like this myself (clear memcached on startup,
>>> in my case) and couldn't figure out a good way.
>>>
>>> There are a couple of issues. One is: what do you mean by startup? It's
>>> fairly well defined if you're running web2py with Rocket as the server, but
>>> in a wsgi environment, especially one that might have multiple processes,
>>> it's a lot less clear.
>>>
>>> Another is that the built-in cron mechanism was badly broken. ISTR
>>> seeing some fixes from Massimo go by recently, but I didn't look closely at
>>> what was going on.
>>>
>>> So: what are you really trying to do? In what way is it a problem to run
>>> on the first request after startup (whatever startup is) rather than at
>>> startup itself? If you're initializing the database, can you check that
>>> it's already initialized? (Yes, overhead, but would it work logically?)
>>>
>>>
>>> I've tried a lot of permutations for setting a script at the start with
>>> cron and I'm now convinced that the functionality is either broken or so
>>> obscure that most non-devs users that rely on the documentation alone won't
>>> be able to make it work (at least, not without a major head ache).
>>>
>>> Besides, it seems from the documentation that cron is being phased out
>>> anyways, so I'm thinking it would be nice to provide some sort of hook for
>>> users to insert a script at startup with access to the models instead.
>>>
>>> Inserting "from gluon import *" in the module won't work either since
>>> the db handles are not defined in gluon, but in the models.
>>>
>>> I'm guessing that maybe the entire database definition could be moved to
>>> a module instead (I'd have to reflect on all the implications of this
>>> though it would definitely increase the performance at the cost of having
>>> to reboot when you want to modify the db's structure), but that's not the
>>> default out of the box architecture.
>>>
>>> Either way, inserting the script into a module that is included in a
>>> controller is not a 100% satisfying solution since the script is actually
>>> executed on the first page request and not really at startup.
>>>
>>> On Wednesday, 28 November 2012 12:55:00 UTC-5, Magnitus wrote:
>>>>
>>>> After some reflection, I suppose I could write a wrapper script that
>>>> first runs python web2py.py --import_models --shell=<MyController>/default
>>>> --run="<My Script>" and then starts web2py normally when using web2py
>>>> standalone.
>>>>
>>>> That is one workaround.
>>>>
>>>> On Wednesday, 28 November 2012 11:19:01 UTC-5, Magnitus wrote:
>>>>>
>>>>> Hi gents,
>>>>>
>>>>> I have been populating my databases with groups (for access control)
>>>>> manually for a while and then, switched to running a script that does it,
>>>>> though I still run the script manually in the web2py context.
>>>>>
>>>>> Now, I'd like to programmatically implement this in a sensible way (in
>>>>> a way that runs by itself and doesn't need human intervention at all).
>>>>>
>>>>> Obviously, you could just put the logic that does this in a model, but
>>>>> I don't like the idea of needlessly increasing the size of my models with
>>>>> this or running the logic to check the existance of and generate groups
>>>>> for
>>>>> each request.
>>>>>
>>>>> Another alternative would be to put the logic in a module that would
>>>>> be imported (with the idea that it would run on the first import which is
>>>>> done only once), but then imported modules don't have access to the
>>>>> web2py
>>>>> objects (notably, my database handle) unless you pass them on as
>>>>> parameters
>>>>> which you cannot do in a module's global scope when it is first included.
>>>>>
>>>>> This leaves me with cron and running it as a cron job at boot time.
>>>>> However, it isn't working.
>>>>>
>>>>> I tried running my script using the web2py context (ie, python
>>>>> web2py.py --import_models --shell=<MyController>/default --run="<My
>>>>> Script>") and it ran fine and generated the db entries I wanted.
>>>>>
>>>>> Then, I started my web2py execution with cron activated (ie, python
>>>>> web2py.py -Y).
>>>>>
>>>>> I tried the two varations portayed in the example for the crontab
>>>>> ("@reboot * * * * root *applications/<My App>/cron/Generate_groups.py"
>>>>> and
>>>>> the shorter "@reboot root *applications/<My App>/cron/Generate_groups.py).
>>>>>
>>>>> And yes, my db transactions end with a db.commit().
>>>>>
>>>>> I've been banging my head against this for two hours and while I'm
>>>>> sure whatever is wrong with this will come to me over time, this is
>>>>> definitely a case of sooner would be better than later.
>>>>>
>>>>> Insights would be appreciated.
>>>>>
>>>>
>>> --
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>> --
>>
>>
>>
>>
>>
>>
>>
> --
>
>
>
>
>
>
>
--
