More DAL testing... It is interesting, in that the first time the dal.py code is executed everything works just fine, however on the very next request I receive this trackback.
http://paste.pocoo.org/show/214476/ sql.py works just fine for many requests. I am obviously using the dal in an environment it was not intended, however sql.py is working so I thought I should mention it. -- Thadeus On Sun, Apr 11, 2010 at 12:02 AM, mdipierro <mdipie...@cs.depaul.edu> wrote: > First of all thank you for your through tests into the DAL. > > I worked into this specific issue and I fixed the discrepancy. It > deserves an explanation anyway. Here is a minimal example to reproduce > the problem: > > code > db.define_table('b', Field('b')) > db.define_table('c', Field('d')) > print db()._select(db.b.ALL, db.c.ALL) > > output with sql.py > SELECT b.id, b.b, c.id, c.d FROM c, b; > > output with dal.py (before fix in trunk) > SELECT b.id, b.b, c.id, c.d FROM b; > > While only the latter fails, they are both wrong because the original > query is wrong. It is doing an inner join but it is missing a link > between table. sql.py assumes you want a cartesian product but that is > not a good idea because the number of records blows up. > dal.py did assumes anything (after the fix behaves as sql.py). > > In your code you use this as > > if len(db().select(db.person.ALL, db.dog_type.ALL, db.dog.ALL))==0 > > and if you have 3000 records (1000 in each table) this returns 10^9 > rows. This runs out out RAM. I think you simply want > > if db(db.person.id)count()+db(db.dog_type.id).count() > +db(db.dog.ALL).count(): > > > On Apr 8, 2:57 pm, Thadeus Burgess <thade...@thadeusb.com> wrote: >> Massimo, I am attaching a application that can replicate this issue. >> >> Works fine on sql.py >> >> Breaks on dal.py >> >> This application is designed after my big application, the represent >> functions HAVE to stay the same, there cannot be ANY code edits to >> them! >> >> Make sure to run with sql.py first to confirm that it IS working >> correctly. THEN copy dal.py to sql.py, restart web2py, and then >> confirm that it no longer works. >> >> -Thadeus >> >> On Thu, Apr 8, 2010 at 10:12 AM, Thadeus Burgess <thade...@thadeusb.com> >> wrote: >> > Agreed. Lets narrow this down, I will see if I can replicate the issue >> > on a smaller scale. >> >> > -Thadeus >> >> > On Wed, Apr 7, 2010 at 11:08 PM, Massimo Di Pierro >> > <mdipie...@cs.depaul.edu> wrote: >> >> I guess my question is " why does it not throw an exception in sql.py?" It >> >> should since >> >> >>>> db.dog.owner.represent = lambda value: "%s" % db.person[value].name >> >> >> when called with value==None should always result in the error you see in >> >> dal.py. >> >> >> Can you help me debug this? My problem is not why it does not work with >> >> dal.py (that is the part I understand). My problem is how is it that it >> >> works with sql.py? >> >> >> On Apr 7, 2010, at 10:58 PM, Thadeus Burgess wrote: >> >> >>> I am not using different datasets, or using different queries. >> >> >>> So here we go, let me explain in every minuet detail this process. >> >> >>>>>> cd ~ >> >>>>>> hg clonehttps://web2py.googlecode.com/hgweb2py >> >>>>>> cd web2py >> >>>>>> ln -s ~/path/to/my/application applications/pms >> >>>>>> python web2py.py -a <recycle> >> >> >>> *go to 127.0.0.1:8000/pms/default/index >> >>> *everything works perfectly, I see the page as it should be with >> >>> records in place, providing the proper "names" (this is only three >> >>> records, I KNOW they have values) >> >> >>>>>> kill -SIGTERM <web2py process id> >> >>>>>> cd gluon/ >> >>>>>> mv sql.py sql.bak.py >> >>>>>> ln -s dal.py sql.py >> >>>>>> cd .. >> >>>>>> python web2py.py -a <recycle> >> >> >>> *go to 127.0.0.1:8000/pms/default/index >> >>> *receive stacktrace. >> >> >>>>>> kill -SIGTERM <web2py process id> >> >>>>>> cd gluon/ >> >>>>>> rm sql.py >> >>>>>> mv sql.bak.py sql.py >> >>>>>> cd .. >> >>>>>> python web2py.py -a <recycle> >> >> >>> *go to 127.0.0.1:8000/pms/default/index >> >>> *behold, everything works perfectly. >> >> >>> Even IF, and I say IF my dataset has a bad reference to a None record, >> >>> and it executes just fine with sql.py, it should execute with dal.py >> >>> exactly the same, it should not be throwing the exception. >> >> >>> -Thadeus >> >> >>> On Wed, Apr 7, 2010 at 10:10 PM, Massimo Di Pierro >> >>> <mdipie...@cs.depaul.edu> wrote: >> >> >>>> I assume it is >> >> >>>>> db.dog.owner.represent = lambda value: "%s" % db.person[value].name >> >> >>>> The stacktrace you get is because one of your records has owner==None >> >>>> hence >> >>>> db.person[value] is also None and None has no .name. It is not a bug. >> >>>> You >> >>>> are doing different queries or using different datasets. >> >> >>>> Anyway if you do >> >> >>>>> db.define_table('person', Field('name'), format='%(name)s') >> >>>>> db.define_table('dog', Field('nickname'), Field('owner', db.person)) >> >> >>>> the requires is set automatically and should take care of this >> >>>> exception. >> >> >>>> On Apr 7, 2010, at 10:01 PM, Thadeus Burgess wrote: >> >> >>>>> I am defining this function. >> >> >>>>> db.define_table('person', Field('name')) >> >>>>> db.define_table('dog', Field('nickname'), Field('owner', db.person)) >> >> >>>>> db.dog.owner.represent = lambda value: "%s" % db.owner[value].name >> >> >>>>> This works in sql.py >> >> >>>>> This does not work in dal.py >> >> >>>>> You said you made changes, I tested, works just fine on sql.py, and >> >>>>> has been for quite a while, however I get that stacktrace when I use >> >>>>> dal.py. >> >> >>>>> -Thadeus >> >> >>>>> On Wed, Apr 7, 2010 at 8:34 PM, mdipierro <mdipie...@cs.depaul.edu> >> >>>>> wrote: >> >> >>>>>> I guess I do not understand. Who defined this represent function? is >> >>>>>> it not in your code? Are you saying this is a bug in sql.py? >> >> >>>>>> On Apr 7, 7:48 pm, Thadeus Burgess <thade...@thadeusb.com> wrote: >> >> >>>>>>> No. >> >> >>>>>>> For dal.py to be accepted it has to work exactly like sql.py >> >> >>>>>>> Therefore I cannot make ANY code changes if we want a new DAL. >> >> >>>>>>> -Thadeus >> >> >>>>>>> On Wed, Apr 7, 2010 at 5:31 PM, mdipierro <mdipie...@cs.depaul.edu> >> >>>>>>> wrote: >> >> >>>>>>>> This >> >> >>>>>>>> db.TableA.id_TableB.represent = lambda value: "%s" % >> >>>>>>>> db.TableB[value].name >> >> >>>>>>>> should be >> >> >>>>>>>> db.TableA.id_TableB.represent = lambda value: (db.TableB[value] or >> >>>>>>>> {}).get('name','anonymous') >> >> >>>>>>>> On Apr 7, 1:36 pm, Thadeus Burgess <thade...@thadeusb.com> wrote: >> >> >>>>>>>>> Traceback (most recent call last): >> >>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/restricted.py", line >> >>>>>>>>> 173, in restricted >> >>>>>>>>> exec ccode in environment >> >>>>>>>>> File >> >> >>>>>>>>> "/home/tburgess/Applications/web2py/applications/pms/views/default/index.html", >> >>>>>>>>> line 62, in <module> >> >>>>>>>>> <thead> >> >>>>>>>>> File >> >> >>>>>>>>> "/home/tburgess/Applications/web2py/applications/pms/models/common.py", >> >>>>>>>>> line 197, in <lambda> >> >>>>>>>>> db.TableA.id_TableB.represent = lambda value: "%s" % >> >>>>>>>>> db.TableB[value].name >> >>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/sql.py", line 2402, >> >>>>>>>>> in __getitem__ >> >>>>>>>>> return self._db(self.id == key).select().first() >> >>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/sql.py", line 3276, >> >>>>>>>>> in >> >>>>>>>>> select >> >>>>>>>>> return self._db._adapter.select(self._query,*fields,**attributes) >> >>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/sql.py", line 507, >> >>>>>>>>> in >> >>>>>>>>> select >> >>>>>>>>> query = self.SELECT(query,*fields, **attributes) >> >>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/sql.py", line 433, >> >>>>>>>>> in >> >>>>>>>>> SELECT >> >>>>>>>>> for field in self.db[table]: >> >>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/sql.py", line 2102, >> >>>>>>>>> in __getitem__ >> >>>>>>>>> return dict.__getitem__(self, str(key)) >> >>>>>>>>> KeyError: 'None' >> >> >>>>>>>>> -Thadeus >> >> >>>>>>>>> On Mon, Apr 5, 2010 at 6:00 PM, mdipierro <mdipie...@cs.depaul.edu> >> >>>>>>>>> wrote: >> >> >>>>>>>>>> uploading fix. please check it. Thanks for testing this. >> >> >>>>>>>>>> On Apr 5, 3:59 pm, Thadeus Burgess <thade...@thadeusb.com> wrote: >> >> >>>>>>>>>>> Traceback (most recent call last): >> >>>>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/restricted.py", >> >>>>>>>>>>> line >> >>>>>>>>>>> 173, in restricted >> >>>>>>>>>>> exec ccode in environment >> >>>>>>>>>>> File >> >> >>>>>>>>>>> "/home/tburgess/Applications/web2py/applications/pms/controllers/default.py", >> >>>>>>>>>>> line 278, in <module> >> >>>>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/globals.py", line >> >>>>>>>>>>> 96, >> >>>>>>>>>>> in <lambda> >> >>>>>>>>>>> self._caller = lambda f: f() >> >>>>>>>>>>> File >> >> >>>>>>>>>>> "/home/tburgess/Applications/web2py/applications/pms/controllers/default.py", >> >>>>>>>>>>> line 76, in duplicates >> >>>>>>>>>>> orderby=db.potentials.A >> >>>>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/sql.py", line >> >>>>>>>>>>> 3258, >> >>>>>>>>>>> in select >> >>>>>>>>>>> return >> >>>>>>>>>>> self._db._adapter.select(self._query,*fields,**attributes) >> >>>>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/sql.py", line 512, >> >>>>>>>>>>> in select >> >>>>>>>>>>> return self.parse(rows,self._colnames) >> >>>>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/sql.py", line 706, >> >>>>>>>>>>> in parse >> >>>>>>>>>>> (rid._table, rid._record) = (db[referee], None) >> >>>>>>>>>>> NameError: global name 'db' is not defined >> >> >>>>>>>>>>> -Thadeus >> >> >>>>>>>>>>> On Mon, Apr 5, 2010 at 3:01 PM, mdipierro >> >>>>>>>>>>> <mdipie...@cs.depaul.edu> >> >>>>>>>>>>> wrote: >> >> >>>>>>>>>>>> try now >> >> >>>>>>>>>>>> On Apr 5, 2:30 pm, Thadeus Burgess <thade...@thadeusb.com> >> >>>>>>>>>>>> wrote: >> >> >>>>>>>>>>>>> Now I get this for reference field >> >> >>>>>>>>>>>>> Traceback (most recent call last): >> >>>>>>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/restricted.py", >> >>>>>>>>>>>>> line >> >>>>>>>>>>>>> 173, in restricted >> >>>>>>>>>>>>> exec ccode in environment >> >>>>>>>>>>>>> File >> >> >>>>>>>>>>>>> "/home/tburgess/Applications/web2py/applications/pms/views/default/index.html", >> >>>>>>>>>>>>> line 84, in <module> >> >>>>>>>>>>>>> {{=db.table[column].represent(d.B[column])}} >> >>>>>>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/sql.py", line >> >>>>>>>>>>>>> 2210, >> >>>>>>>>>>>>> in __getattr__ >> >>>>>>>>>>>>> self.__allocate() >> >>>>>>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/sql.py", line >> >>>>>>>>>>>>> 2203, >> >>>>>>>>>>>>> in __allocate >> >>>>>>>>>>>>> self._record = self._table[int(self)] >> >>>>>>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/sql.py", line >> >>>>>>>>>>>>> 2384, >> >>>>>>>>>>>>> in __getitem__ >> >>>>>>>>>>>>> return self._db(self.id == key).select().first() >> >>>>>>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/sql.py", line >> >>>>>>>>>>>>> 3258, in select >> >>>>>>>>>>>>> return >> >>>>>>>>>>>>> self._db._adapter.select(self._query,*fields,**attributes) >> >>>>>>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/sql.py", line >> >>>>>>>>>>>>> 512, >> >>>>>>>>>>>>> in select >> >>>>>>>>>>>>> return self.parse(rows,self._colnames) >> >>>>>>>>>>>>> File "/home/tburgess/Applications/web2py/gluon/sql.py", line >> >>>>>>>>>>>>> 707, >> >>>>>>>>>>>>> in parse >> >>>>>>>>>>>>> colset[fieldname] = rid = Reference(value) >> >>>>>>>>>>>>> TypeError: int() argument must be a string or a number, not >> >>>>>>>>>>>>> 'NoneType' >> >> >>>>>>>>>>>>> -Thadeus >> >> >>>>>>>>>>>>> On Mon, Apr 5, 2010 at 2:20 PM, mdipierro >> >>>>>>>>>>>>> <mdipie...@cs.depaul.edu> wrote: >> >> >>>>>>>>>>>>>> I fixed the former. I did not fix the latter. It is a known >> >>>>>>>>>>>>>> problem >> >>>>>>>>>>>>>> with the >> >> ... >> >> read more » >> >> web2py.app.test_dalpy.w2p >> 110KViewDownload > > > -- > To unsubscribe, reply using "remove me" as the subject. >
