Result: Fixed by upgrading. I was seeing this bug:
http://code.google.com/p/web2py/issues/detail?id=345
However, virtualfields still take more time than they should. My
selects take 2-3x longer with virtualfields enabled than without. I
implemented a little hack in the dal that adds methods to rows with
only a 10% overhead (instead of 200-300%) and can share that if
anyone's interested.
On Aug 8, 8:38 pm, Michael Toomim <too...@gmail.com> wrote:
> It turns out the speed problem is REALLY bad. I have a table with
> virtualfields of 14,000 rows. When I run raw sql:
>
> a = db.executesql('select * from people;')
>
> ...the query returns in 121ms. But when I run it through the DAL on
> only a subset of the data:
>
> a = db(db.people.id > 0).select(limitby=(0,1000))
>
> ...it returns in 141096.431ms. That's... 141 seconds. So 1000x longer
> on .1 of the database.
>
> My virtualfields are all lazy functions. I'm looking into what's
> causing it and will report back when I find out. It seems it might
> have something to do with the lazy decorator func because when I hit C-
> c the code is often stuck there... inside import copy or something.
>
> def lazy(f):
> def g(self,f=f):
> import copy
> self=copy.copy(self)
> return lambda *a,**b: f(self,*a,**b)
> return g
>
> Anyway, I'll send an update when I have more info.
>
> On Aug 2, 3:03 pm, MichaelToomim<too...@gmail.com> wrote:
>
>
>
>
>
>
>
> > That's way better syntax! Great idea!
>
> > On Aug 2, 2011, at 2:31 AM, Massimo Di Pierro wrote:
>
> > > We need to work on the speed. This can perhaps help the syntax:
>
> > > db=DAL()
> > > db.define_table('a',Field('b','integer'))
> > > for i in range(10):
> > > db.a.insert(b=i)
>
> > > def lazy(f):
> > > def g(self,f=f):
> > > import copy
> > > self=copy.copy(self)
> > > return lambda *a,**b: f(self,*a,**b)
> > > return g
>
> > > class Scale:
> > > @lazy
> > > def c(self,scale=1):
> > > return self.a.b*scale
>
> > > db.a.virtualfields.append(Scale())
> > > for row in db(db.a).select():
> > > print row.b, row.c(1), row.c(2), row.c(3)
>
> > > On Aug 1, 3:10 pm, MichaelToomim<too...@gmail.com> wrote:
> > >> Maybe it helps for me to explain my use-case. I mainly use virtual
> > >> fields as lazy methods, to help traverse related tables. I was actually
> > >> surprised that lazy evaluation wasn't the default. I noticed a few
> > >> implications of this:
> > >> - Large queries are slowed byvirtualfields, even if they won't be
> > >> needed, esp if they query db
> > >> - My definitions forvirtualfieldsaren't as clean as they could be,
> > >> because I have many nested "lazy" funcs in the class definition
> > >> - We can't serialize all objects intosessionvariables
>
> > >> So really I'm just using this because it's a nicer notation to call
> > >> row.otherthing() instead of getotherthing(row). Maybe I really want some
> > >> different feature here?
>
> > >> On Aug 1, 2011, at 5:40 AM, Anthony Bastardi wrote:
>
> > >>> Note, after looking at this some more, Massimo recalled that the reason
> > >>> auth_user virtual fields were excluded from auth.user (and therefore
> > >>> from saving in thesession) is because some virtual fields are objects
> > >>> that cannot be pickled and therefore cannot be serialized to store in
> > >>> thesession. So, we're thinking of either creating an option to store
> > >>> auth_user virutual fields in auth.user, or maybe testing to make sure
> > >>> the virtual fields can be pickled, and excluding them if not.
>
> > >>> Anthony
>
> > >>> On Mon, Aug 1, 2011 at 5:30 AM, MichaelToomim<too...@cs.washington.edu>
> > >>> wrote:
> > >>> Awesome! I did not know there was an issue submission system.
>
> > >>> On Jul 30, 2011, at 7:02 AM, Anthony wrote:
>
> > >>>> An issue has been submitted, and this should be corrected soon.
>
> > >>>> Anthony
>
> > >>>> On Friday, July 29, 2011 9:57:30 PM UTC-4, Anthony wrote:
> > >>>> auth.user is Storage(table_user._filter_fields(user, id=True)). The
> > >>>> _filter_fields method of the auth_user table only selects actual table
> > >>>> fields, not virtual fields, so auth.user will not include any virtual
> > >>>> fields. Perhaps this should be changed.
>
> > >>>> Anthony
>
> > >>>> On Friday, July 29, 2011 9:05:39 PM UTC-4, MichaelToomimwrote:
> > >>>> I think I found a bug invirtualfields. I have the following
> > >>>> controller:
>
> > >>>> def posts():
> > >>>> user =session.auth.user
> > >>>> n = user.name # returns None
>
> > >>>> Where "person" is defined as a virtualfield on user:
>
> > >>>> class Users():
> > >>>> def name(self):
> > >>>> return self.users.first_name + ' ' + self.users.last_name
> > >>>> db.users.virtualfields.append(Users())
>
> > >>>> The problem is that user.name returns None, because apparently the
> > >>>> virtualfield isn't loaded into thesessionvariable of user.
>
> > >>>> I made this work with the following modification to the controller:
>
> > >>>> def posts():
> > >>>> user = db.users[session.auth.user.id]
> > >>>> n = user.name # returns the user name correctly!
>
> > >>>> I just had to refetch the user from the database.
