Holy cow, you're right!
I've had so many things on my mind that I totally forgot about that!
Guess that will speed up my production app a little then...
On Friday, 30 November 2012 13:06:34 UTC-5, Jonathan Lundell wrote:
>
> On 30 Nov 2012, at 9:54 AM, Magnitus <eric_va...@yahoo.ca <javascript:>>
> wrote:
>
> It is not executed, but given that models are not "compiled" into bytecode
> as far as I know, won't it have to be interpreted (incurring a performance
> penalty for the interpreter having to parse additional code)?
>
>
> Ordinarily, in production, your files would be precompiled. I think you'd
> be hard pressed to measure the additional overhead of a few extra lines in
> your model.
>
>
> On Friday, 30 November 2012 12:50:33 UTC-5, Jonathan Lundell wrote:
>>
>> On 30 Nov 2012, at 9:38 AM, Magnitus <eric_va...@yahoo.ca> wrote:
>>
>> What you are suggesting addresses the concurrency access problem (which
>> could be resolved with the DB since it locks or with your suggestion).
>>
>> However, if you stick the logic in your model, you still have the problem
>> of the logic appearing on every request.
>>
>> While you do speed up the checking with the "Test & set" instead of DB
>> access, eliminating the disk read, you do burden your model with additional
>> code bloat.
>>
>> I suppose that could be mitigated by inserting your script logic into a
>> module and passing the db as an argument to it.
>>
>>
>> The overhead to check global_settings is negligibly more than an ordinary
>> dict access, and except for the first request, that's all you'll incur. I'm
>> pretty sure that's quite a bit less than an import of a module that's
>> already imported (since it still has to check whether the module is already
>> there).
>>
>> It's true that the db code will still be there, but it won't be executed.
>>
>>
>> On Thursday, 29 November 2012 14:00:05 UTC-5, Jonathan Lundell wrote:
>>>
>>> On 29 Nov 2012, at 10:22 AM, Magnitus <eric_va...@yahoo.ca> wrote:
>>>
>>> >There are a couple of issues. One is: what do you mean by startup? It's
>>> fairly well defined if you're running web2py with Rocket as the server, but
>>> in a wsgi environment, especially one that might have multiple processes,
>>> it's a lot less clear.
>>>
>>> Yes and no.
>>>
>>> Obviously, a task that needs to run at startup would have to be robust,
>>> because you might start and stop your server multiple times (for example if
>>> it creates groups, it would have to check if the groups already exist
>>> before creating them).
>>>
>>> In that context, it isn't too much of a problem if it is executed
>>> multiple times if wsgi spawns multiple processes given that it is an
>>> overhead only when the server starts.
>>>
>>> However, if you absolutely need to make sure that the task itself runs
>>> only once even with multiple processes spawning, it seems like such
>>> synchronization could be handled in the DB (either as a feature of the
>>> framework or by the scripter himself, it's not too big of an overhead) or
>>> just by using another interprocess synchronisation mechanism that modern
>>> OSes provide.
>>>
>>> > So: what are you really trying to do? In what way is it a problem to
>>> run on the first request after startup (whatever startup is) rather than at
>>> startup itself? If you're initializing the database, can you check that
>>> it's already initialized? (Yes, overhead, but would it work logically?)
>>>
>>> I'm sure it's not THAT big of a performance overhead, but if possible, I
>>> prefer that tasks that only need to be executed at startup be only executed
>>> at startup.
>>>
>>> Not only does it run faster, but intuitively, I consider it to be better
>>> design as it modularize your code more since it doesn't mix up tasks that
>>> need to be executed at startup only with tasks that need to be executed
>>> with every request.
>>>
>>>
>>> How about something along these lines: set a flag in
>>> gluon.settings.global_settings, using an app-unique key (because
>>> global_settings should but doesn't have a per-application namespace you
>>> want something like 'com.yourdomain.yourapp' or the like). Test & set the
>>> flag early in your model, and do your db init conditional on the model not
>>> being set, using db semantics to do it once, since you could have updates
>>> from multiple processes. A straightforward mechanism (I think?) would be to
>>> just do an insert with some appropriate field marked unique, and ignore
>>> insert-collision exceptions.
>>>
>>>
>>> On Thursday, 29 November 2012 10:44:17 UTC-5, Jonathan Lundell wrote:
>>>>
>>>> On 29 Nov 2012, at 6:41 AM, Magnitus <eric_va...@yahoo.ca> wrote:
>>>>
>>>> I'd be curious to know if the lack of feedback on this thread so far is
>>>> due to the devs being incredibly busy or web2py lacking a genuine working
>>>> mechanism to execute scripts that have access to the models at startup
>>>> only.
>>>>
>>>>
>>>> I've tried to do something like this myself (clear memcached on
>>>> startup, in my case) and couldn't figure out a good way.
>>>>
>>>> There are a couple of issues. One is: what do you mean by startup? It's
>>>> fairly well defined if you're running web2py with Rocket as the server,
>>>> but
>>>> in a wsgi environment, especially one that might have multiple processes,
>>>> it's a lot less clear.
>>>>
>>>> Another is that the built-in cron mechanism was badly broken. ISTR
>>>> seeing some fixes from Massimo go by recently, but I didn't look closely
>>>> at
>>>> what was going on.
>>>>
>>>> So: what are you really trying to do? In what way is it a problem to
>>>> run on the first request after startup (whatever startup is) rather than
>>>> at
>>>> startup itself? If you're initializing the database, can you check that
>>>> it's already initialized? (Yes, overhead, but would it work logically?)
>>>>
>>>>
>>>> I've tried a lot of permutations for setting a script at the start with
>>>> cron and I'm now convinced that the functionality is either broken or so
>>>> obscure that most non-devs users that rely on the documentation alone
>>>> won't
>>>> be able to make it work (at least, not without a major head ache).
>>>>
>>>> Besides, it seems from the documentation that cron is being phased out
>>>> anyways, so I'm thinking it would be nice to provide some sort of hook for
>>>> users to insert a script at startup with access to the models instead.
>>>>
>>>> Inserting "from gluon import *" in the module won't work either since
>>>> the db handles are not defined in gluon, but in the models.
>>>>
>>>> I'm guessing that maybe the entire database definition could be moved
>>>> to a module instead (I'd have to reflect on all the implications of this
>>>> though it would definitely increase the performance at the cost of having
>>>> to reboot when you want to modify the db's structure), but that's not the
>>>> default out of the box architecture.
>>>>
>>>> Either way, inserting the script into a module that is included in a
>>>> controller is not a 100% satisfying solution since the script is actually
>>>> executed on the first page request and not really at startup.
>>>>
>>>> On Wednesday, 28 November 2012 12:55:00 UTC-5, Magnitus wrote:
>>>>>
>>>>> After some reflection, I suppose I could write a wrapper script that
>>>>> first runs python web2py.py --import_models
>>>>> --shell=<MyController>/default
>>>>> --run="<My Script>" and then starts web2py normally when using web2py
>>>>> standalone.
>>>>>
>>>>> That is one workaround.
>>>>>
>>>>> On Wednesday, 28 November 2012 11:19:01 UTC-5, Magnitus wrote:
>>>>>>
>>>>>> Hi gents,
>>>>>>
>>>>>> I have been populating my databases with groups (for access control)
>>>>>> manually for a while and then, switched to running a script that does
>>>>>> it,
>>>>>> though I still run the script manually in the web2py context.
>>>>>>
>>>>>> Now, I'd like to programmatically implement this in a sensible way
>>>>>> (in a way that runs by itself and doesn't need human intervention at
>>>>>> all).
>>>>>>
>>>>>> Obviously, you could just put the logic that does this in a model,
>>>>>> but I don't like the idea of needlessly increasing the size of my models
>>>>>> with this or running the logic to check the existance of and generate
>>>>>> groups for each request.
>>>>>>
>>>>>> Another alternative would be to put the logic in a module that would
>>>>>> be imported (with the idea that it would run on the first import which
>>>>>> is
>>>>>> done only once), but then imported modules don't have access to the
>>>>>> web2py
>>>>>> objects (notably, my database handle) unless you pass them on as
>>>>>> parameters
>>>>>> which you cannot do in a module's global scope when it is first included.
>>>>>>
>>>>>> This leaves me with cron and running it as a cron job at boot time.
>>>>>> However, it isn't working.
>>>>>>
>>>>>> I tried running my script using the web2py context (ie, python
>>>>>> web2py.py --import_models --shell=<MyController>/default --run="<My
>>>>>> Script>") and it ran fine and generated the db entries I wanted.
>>>>>>
>>>>>> Then, I started my web2py execution with cron activated (ie, python
>>>>>> web2py.py -Y).
>>>>>>
>>>>>> I tried the two varations portayed in the example for the crontab
>>>>>> ("@reboot * * * * root *applications/<My App>/cron/Generate_groups.py"
>>>>>> and
>>>>>> the shorter "@reboot root *applications/<My
>>>>>> App>/cron/Generate_groups.py).
>>>>>>
>>>>>> And yes, my db transactions end with a db.commit().
>>>>>>
>>>>>> I've been banging my head against this for two hours and while I'm
>>>>>> sure whatever is wrong with this will come to me over time, this is
>>>>>> definitely a case of sooner would be better than later.
>>>>>>
>>>>>> Insights would be appreciated.
>>>>>>
>>>>>
>>>> --
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>> --
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>> --
>>
>>
>>
>>
>>
>>
>>
> --
>
>
>
>
>
>
>
--
