Thanks for this test Alex.
Massimo
On Aug 30, 1:45 pm, what_ho <a...@viovi.com> wrote:
> Hi all - just had time this weekend to look further into this.
>
> Thank you to suggestions from Iceberg, Yarko and Massimo. I have tried
> putting table definitions into a module, and also run some timing
> tests against a Postgres database
>
> As a note, for my previous tests coming out at 12ms page response,
> those same previous test now come out at 10ms on my computer .. I
> think just in picking up latest Ubuntu patches etc. something must
> have changed in my configuration, anyway all the figures below are
> just a rough guide.
>
> Here are the changes I made:
>
> db and mail object creation now in modules\application.py.
> These objects are created once then shared.
>
> Setting 'ENABLE_APPLICATION_SCOPE = False' in application.py switches
> back to standard behaviour (separate db and mail object on every page
> request)
>
> auth,crud,service objects are dependent on the current page request,
> so I left these in models\db.py.
>
> One exception is auth.define_tables - this is run in modules
> \application.py. Usually runs once only unless
> ENABLE_APPLICATION_SCOPE is False, then auth.define_tables runs on
> every page request.
>
> Postgres version is 8.4.0-2 installed on same machine web2py is
> running on. Computer set-up same as for previous test (except for
> Ubuntu patches mentioned above), the test is to request the index page
> created by default for a new web2py application.
>
> Running 1000 page requests with httperf tool:
>
> postgres, compiled, migrate=false, application scope objects disabled,
> pool_size parameter not set:
> 58.2ms average page response time
>
> postgres, compiled, migrate=false, application scope objects disabled,
> pool_size=100:
> 10.9ms average page response time
>
> postgres, compiled, migrate=false, application scope objects
> enabled,pool_size=100:
> 5.2ms average page response time
>
> sqllite, compiled, migrate=false, application scope objects disabled:
> 10.1ms average page response time
>
> sqllite, compiled, migrate=false, application scope objects enabled:
> 5.4ms average page response time
>
> ---------
>
> My summary:
>
> Sharing objects like the database model definition at the website
> application scope does give a speed boost, but it is not as big a
> difference as I thought - I have been pleasantly surprised that the
> existing per-webpage request evaluation of models was already pretty
> fast as it is.
>
> The difference in Postgres with timings when pool_size is not set at
> all is significant : This almost warrants defaulting the pool size for
> this database to 100 anyway, or at least outputting to the log a
> friendly warning to suggest setting a pool size parameter.
>
> I can see some downsides when attempting to use objects at application
> scope:
>
> - I'm not yet sure objects like DAL and Mail will be happy being
> shared at the application level. Could well be locking / threading
> issues, would need to investigate further
>
> - In general in Python, module source code changes do not get picked
> up automatically. During development I would work around this with the
> reload(...) statement
>
> - My example shared module probably needs additional try..catch
> statements to handle if it fails to run, so every webpage request will
> keep on trying to create the shared objects until they have been
> successfully created.
>
> - There may be good reason to refresh these shared objects
> periodically anyway, maybe tie the shared object creation to a cron
> job.
>
> If I could be confident there are no threading issues then using
> shared objects could be an option. But until then I am now happier
> sticking with the standard setup - putting maybe quite complex model
> structures in the regular model folder, these will be evaluated on
> every webpage request, but the site will still run plenty fast - good
> enough for me anyway!
>
> Here is my updated db.py and module source code at the end of this
> message. Just add both to a blank application created by the web2py
> admin.
>
> Cheers,
> - Alex
>
> --
> db.py model file, pulling in shared 'application scope' objects:
>
> # coding: utf8
> from gluon.tools import *
> import applications.perftest.modules.application as app
>
> #Get objects shared between page requests
> app.Helper.get_application_scope(globals())
>
> #Rest of this file sets up per-page model state:
> if not globals().has_key('auth'):
> auth=Auth(globals(),db)
>
> auth.settings.hmac_key='6944f00d-1758-41e4-8fdb-625b0be17e1a'
> auth.settings.mailer=mail # for user email
> verification
> auth.settings.registration_requires_verification = True
> auth.settings.registration_requires_approval = True
> auth.messages.verify_email = \
> 'Click on the linkhttp://.../user/verify_email/%(key)sto verify
> your email'
> crud=Crud(globals(),db)
> crud.settings.auth=auth
>
> #Connect gae session if required
> if request.env.web2py_runtime_gae: session.connect(request, response,
> db=db)
>
> #Set up service for this request
> from gluon.tools import *
> service=Service(globals())
>
> ----------------------------------------
> application.py module file:
>
> # coding: utf8
>
> from gluon.storage import Storage
> from gluon.sql import SQLDB, SQLField, DAL, Field
> from gluon.tools import *
> import thread
>
> ENABLE_APPLICATION_SCOPE = True
>
> class Helper(object):
>
> singleton = None
>
> def __init__(self, environment):
>
> request = environment['request']
> self.storage = Storage()
> self.storage.db = self.get_db(request)
> self.storage.mail = self.get_mail()
>
> @staticmethod
> def get_application_scope(globals):
>
> if ENABLE_APPLICATION_SCOPE:
>
> if Helper.singleton is None:
> locker = thread.allocate_lock()
> locker.acquire()
> Helper.singleton = Helper(globals)
> Helper.singleton.init_auth(globals);
> locker.release()
>
> globals.update(Helper.singleton.storage)
> else:
> request_scope = Helper(globals)
> request_scope.storage.auth = request_scope.init_auth
> (globals)
> globals.update(request_scope.storage)
>
> def get_db(self, request):
>
> if request.env.web2py_runtime_gae:
> db = DAL('gae')
> else:
> #db = DAL('sqlite://storage.sqlite')
> db=DAL('postgres://testuser:t...@localhost:5432/web2py',
> pool_size=100)
>
> for i in range(1, 5):
> db.define_table('perftest' + str(i),
> SQLField('string1','string'),
> SQLField('text1','text'),
> SQLField('blob1','blob'),
> SQLField('password1','password'),
> SQLField('upload1','upload'),
> SQLField('boolean1','boolean'),
> SQLField('integer1','integer'),
> SQLField('double1','double'),
> SQLField('date1','date'),
> SQLField('time1','time'),
> SQLField('datetime1','datetime'),migrate=False)
>
> return db
>
> def get_mail(self):
> mail=Mail() # mailer
> mail.settings.server='smtp.gmail.com:587' # your SMTP
> server
> mail.settings.sender='....@gmail.com' # your email
> mail.settings.login='username:password' # your
> credentials or None
> return mail
>
> #Just used to initialise auth database tables
> def init_auth(self, globals):
> auth=Auth(globals, self.storage.db)
> auth.define_tables(migrate=False)
> return auth
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To post to this group, send email to web2py@googlegroups.com
To unsubscribe from this group, send email to
web2py+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/web2py?hl=en
-~----------~----~----~----~------~----~------~--~---
