let us see it!

On Aug 9, 9:36 pm, Michael Toomim <too...@gmail.com> wrote:
> Result: Fixed by upgrading. I was seeing this 
> bug:http://code.google.com/p/web2py/issues/detail?id=345
>
> However, virtualfields still take more time than they should. My
> selects take 2-3x longer with virtualfields enabled than without. I
> implemented a little hack in the dal that adds methods to rows with
> only a 10% overhead (instead of 200-300%) and can share that if
> anyone's interested.
>
> On Aug 8, 8:38 pm, Michael Toomim <too...@gmail.com> wrote:
>
>
>
>
>
>
>
> > It turns out the speed problem is REALLY bad. I have a table with
> > virtualfields of 14,000 rows. When I run raw sql:
>
> >     a = db.executesql('select * from people;')
>
> > ...the query returns in 121ms. But when I run it through the DAL on
> > only a subset of the data:
>
> >     a = db(db.people.id > 0).select(limitby=(0,1000))
>
> > ...it returns in 141096.431ms. That's... 141 seconds. So 1000x longer
> > on .1 of the database.
>
> > My virtualfields are all lazy functions. I'm looking into what's
> > causing it and will report back when I find out. It seems it might
> > have something to do with the lazy decorator func because when I hit C-
> > c the code is often stuck there... inside import copy or something.
>
> > def lazy(f):
> >    def g(self,f=f):
> >        import copy
> >        self=copy.copy(self)
> >        return lambda *a,**b: f(self,*a,**b)
> >    return g
>
> > Anyway, I'll send an update when I have more info.
>
> > On Aug 2, 3:03 pm, MichaelToomim<too...@gmail.com> wrote:
>
> > > That's way better syntax!  Great idea!
>
> > > On Aug 2, 2011, at 2:31 AM, Massimo Di Pierro wrote:
>
> > > > We need to work on the speed. This can perhaps help the syntax:
>
> > > > db=DAL()
> > > > db.define_table('a',Field('b','integer'))
> > > > for i in range(10):
> > > >    db.a.insert(b=i)
>
> > > > def lazy(f):
> > > >    def g(self,f=f):
> > > >        import copy
> > > >        self=copy.copy(self)
> > > >        return lambda *a,**b: f(self,*a,**b)
> > > >    return g
>
> > > > class Scale:
> > > >    @lazy
> > > >    def c(self,scale=1):
> > > >        return self.a.b*scale
>
> > > > db.a.virtualfields.append(Scale())
> > > > for row in db(db.a).select():
> > > >    print row.b, row.c(1), row.c(2), row.c(3)
>
> > > > On Aug 1, 3:10 pm, MichaelToomim<too...@gmail.com> wrote:
> > > >> Maybe it helps for me to explain my use-case. I mainly use virtual 
> > > >> fields as lazy methods, to help traverse related tables. I was 
> > > >> actually surprised that lazy evaluation wasn't the default. I noticed 
> > > >> a few implications of this:
> > > >>   - Large queries are slowed byvirtualfields, even if they won't be 
> > > >> needed, esp if they query db
> > > >>   - My definitions forvirtualfieldsaren't as clean as they could be, 
> > > >> because I have many nested "lazy" funcs in the class definition
> > > >>   - We can't serialize all objects intosessionvariables
>
> > > >> So really I'm just using this because it's a nicer notation to call 
> > > >> row.otherthing() instead of getotherthing(row). Maybe I really want 
> > > >> some different feature here?
>
> > > >> On Aug 1, 2011, at 5:40 AM, Anthony Bastardi wrote:
>
> > > >>> Note, after looking at this some more, Massimo recalled that the 
> > > >>> reason auth_user virtual fields were excluded from auth.user (and 
> > > >>> therefore from saving in thesession) is because some virtual fields 
> > > >>> are objects that cannot be pickled and therefore cannot be serialized 
> > > >>> to store in thesession. So, we're thinking of either creating an 
> > > >>> option to store auth_user virutual fields in auth.user, or maybe 
> > > >>> testing to make sure the virtual fields can be pickled, and excluding 
> > > >>> them if not.
>
> > > >>> Anthony
>
> > > >>> On Mon, Aug 1, 2011 at 5:30 AM, 
> > > >>> MichaelToomim<too...@cs.washington.edu> wrote:
> > > >>> Awesome! I did not know there was an issue submission system.
>
> > > >>> On Jul 30, 2011, at 7:02 AM, Anthony wrote:
>
> > > >>>> An issue has been submitted, and this should be corrected soon.
>
> > > >>>> Anthony
>
> > > >>>> On Friday, July 29, 2011 9:57:30 PM UTC-4, Anthony wrote:
> > > >>>> auth.user is Storage(table_user._filter_fields(user, id=True)). The 
> > > >>>> _filter_fields method of the auth_user table only selects actual 
> > > >>>> table fields, not virtual fields, so auth.user will not include any 
> > > >>>> virtual fields. Perhaps this should be changed.
>
> > > >>>> Anthony
>
> > > >>>> On Friday, July 29, 2011 9:05:39 PM UTC-4, MichaelToomimwrote:
> > > >>>> I think I found a bug invirtualfields. I have the following
> > > >>>> controller:
>
> > > >>>> def posts():
> > > >>>>     user =session.auth.user
> > > >>>>     n = user.name # returns None
>
> > > >>>> Where "person" is defined as a virtualfield on user:
>
> > > >>>> class Users():
> > > >>>>     def name(self):
> > > >>>>         return self.users.first_name + ' ' + self.users.last_name
> > > >>>> db.users.virtualfields.append(Users())
>
> > > >>>> The problem is that user.name returns None, because apparently the
> > > >>>> virtualfield isn't loaded into thesessionvariable of user.
>
> > > >>>> I made this work with the following modification to the controller:
>
> > > >>>> def posts():
> > > >>>>     user = db.users[session.auth.user.id]
> > > >>>>     n = user.name # returns the user name correctly!
>
> > > >>>> I just had to refetch the user from the database.

Reply via email to