Re: [web2py] Need a little help in code review (a function eating up all memory)

[Bruce Wade]

WOW not a good idea:
for row in db( db.file_properties.id > 0 ).select(

If you have a lot of records that is going to kill your memory.

On Thu, May 10, 2012 at 12:10 AM, szimszon  wrote:

> I wonder if somebody could help me.
>
> The following code has eaten up ~1,5GB ram and after ended successfully it
> is not released :(
>
> file_properties table has 137314 rows.
>
> Model:
> db.define_table( 'files_store',
> Field( 'name', 'string',
> label = 'Fájlnév' ,
> ),
> Field( 'content', 'upload',
> uploadseparate = True,
> autodelete = True,
> label = 'Aláírt fájl',
> ),
> Field( 'status', 'string',
> label = 'Állapot',
> requires = IS_IN_SET( ['empty'
> , 'notsigned', 'senttosign', 'signed'],
>
>   [ 'Üres', 'Aláiratlan', 'Aláírásra elküldve', 'Aláírt'] ),
> represent = lambda r, row:T(r
> )
> ),
> Field( 'state', 'string',
> label = 'Státusz',
> requires = IS_IN_SET( ['ok',
> 'error'],
>
>   [ 'Rendben', 'Hiba'] ),
> represent = lambda r, row:T(r
> )
> ),
> Field( 'data_count', 'integer',
> label = 'Adat bejegyzések',
> requires = IS_NOT_EMPTY(),
> default = 0,
> ),
> Field( 'dirname', 'string',
> label = 'Könyvtár név' ,
> requires = IS_NOT_EMPTY(),
> ),
> Field( 'md5sum', 'string',
> label = T( 'MD5 Sum' ),
> represent = lambda md5sum,junk
> : '[%s]' % md5sum,
> writable = False ),
> auth.signature,
> format = '%(dirname)s/%(name)s (%(id)s)'
>  )
> db.files_store.created_on.readable = True
> db.define_table( 'file_properties',
> Field( 'files_store_id', db.files_store,
> label = 'Aláírt fájl' ,
> ),
> Field( 'kezdo_oldal', 'integer',
> label = 'Számla kezdő oldal',
> requires = IS_NOT_EMPTY(),
> ),
> Field( 'oldalszam', 'integer',
> label = 'Számla oldalainak
> száma',
> requires = IS_NOT_EMPTY(),
> ),
> Field( 'szamla_sorszama', 'string',
> label = 'Számla sorszáma',
> #requires = IS_NOT_IN_DB( db,
> 'file_properties' ),
> unique = True,
> ),
> Field( 'ugyfel_azonosito', 'string',
> label = 'Ügyfél azonosító',
> requires = IS_NOT_EMPTY(),
> ),
> Field( 'folyoszamla_azonosito', 'string',
> label = 'Folyószámla
> azonosító',
> requires = IS_NOT_EMPTY(),
> ),
> Field( 'teljesites_idopontja', 'date',
> label = 'Teljesítés időpontja'
> ,
> requires = IS_DATE(
> '%Y.%m.%d.' ) ),
> auth.signature,
> ),
>
>
>
>
> Controller:
>
> def autoadjust():
> lista = list()
> last_row = None
> next_page_number = 0
> for row in db( db.file_properties.id > 0 ).select(
>
>   orderby = ( db.file_properties.files_sto

Re: [web2py] Need a little help in code review (a function eating up all memory)

[szimszon]

Yes I know but it happens over type the mem usage is linearly growing and 
after the successful execution never released and that is why I ask :(

2012. május 10., csütörtök 9:14:14 UTC+2 időpontban Bruce Wade a következőt 
írta:
>
> WOW not a good idea: 
> for row in db( db.file_properties.id > 0 ).select(
>
> If you have a lot of records that is going to kill your memory.
>
> On Thu, May 10, 2012 at 12:10 AM, szimszon  wrote:
>
>> I wonder if somebody could help me.
>>
>> The following code has eaten up ~1,5GB ram and after ended successfully 
>> it is not released :(
>>
>> file_properties table has 137314 rows.
>>
>> Model:
>> db.define_table( 'files_store',
>> Field( 'name', 'string',
>> label = 'Fájlnév' ,
>> ),
>> Field( 'content', 'upload',
>> uploadseparate = True,
>> autodelete = True,
>> label = 'Aláírt fájl',
>> ),
>> Field( 'status', 'string',
>> label = 'Állapot',
>> requires = IS_IN_SET( [
>> 'empty', 'notsigned', 'senttosign', 'signed'],
>>   
>>   [ 'Üres', 'Aláiratlan', 'Aláírásra elküldve', 'Aláírt'] ),
>> represent = lambda r, row:T(r 
>> )
>> ),
>> Field( 'state', 'string',
>> label = 'Státusz',
>> requires = IS_IN_SET( ['ok', 
>> 'error'],
>>   
>>   [ 'Rendben', 'Hiba'] ),
>> represent = lambda r, row:T(r 
>> )
>> ),
>> Field( 'data_count', 'integer',
>> label = 'Adat bejegyzések',
>> requires = IS_NOT_EMPTY(),
>> default = 0,
>> ),
>> Field( 'dirname', 'string',
>> label = 'Könyvtár név' ,
>> requires = IS_NOT_EMPTY(),
>> ),
>> Field( 'md5sum', 'string',
>> label = T( 'MD5 Sum' ),
>> represent = lambda md5sum,junk
>> : '[%s]' % md5sum,
>> writable = False ),
>> auth.signature,
>> format = '%(dirname)s/%(name)s (%(id)s)'
>>  )
>> db.files_store.created_on.readable = True
>> db.define_table( 'file_properties',
>> Field( 'files_store_id', db.files_store,
>> label = 'Aláírt fájl' ,
>> ),
>> Field( 'kezdo_oldal', 'integer',
>> label = 'Számla kezdő oldal',
>> requires = IS_NOT_EMPTY(),
>> ),
>> Field( 'oldalszam', 'integer',
>> label = 'Számla oldalainak 
>> száma',
>> requires = IS_NOT_EMPTY(),
>> ),
>> Field( 'szamla_sorszama', 'string',
>> label = 'Számla sorszáma',
>> #requires = IS_NOT_IN_DB( 
>> db, 'file_properties' ),
>> unique = True,
>> ),
>> Field( 'ugyfel_azonosito', 'string',
>> label = 'Ügyfél azonosító',
>> requires = IS_NOT_EMPTY(),
>> ),
>> Field( 'folyoszamla_azonosito', 'string',
>> label = 'Folyószámla 
>> azonosító',
>> requires = IS_NOT_EMPTY(),
>> ),
>> Field( 'teljesites_idopontja', 'date',
>> 

Re: [web2py] Need a little help in code review (a function eating up all memory)

[Bruce Wade]

That is how python is. If you want something to clear the memory as soon as
you are done with it you need C++ :D

On Thu, May 10, 2012 at 12:27 AM, szimszon  wrote:

> Yes I know but it happens over type the mem usage is linearly growing and
> after the successful execution never released and that is why I ask :(
>
> 2012. május 10., csütörtök 9:14:14 UTC+2 időpontban Bruce Wade a
> következőt írta:
>
>> WOW not a good idea:
>> for row in db( db.file_**properties.id > 0 ).select(
>>
>> If you have a lot of records that is going to kill your memory.
>>
>> On Thu, May 10, 2012 at 12:10 AM, szimszon  wrote:
>>
>>> I wonder if somebody could help me.
>>>
>>> The following code has eaten up ~1,5GB ram and after ended successfully
>>> it is not released :(
>>>
>>> file_properties table has 137314 rows.
>>>
>>> Model:
>>> db.define_table( 'files_store',
>>> Field( 'name', 'string',
>>> label = 'Fájlnév' ,
>>> ),
>>> Field( 'content', 'upload',
>>> uploadseparate = True,
>>> autodelete = True,
>>> label = 'Aláírt fájl',
>>> ),
>>> Field( 'status', 'string',
>>> label = 'Állapot',
>>> requires = IS_IN_SET( [
>>> 'empty', 'notsigned', 'senttosign', 'signed'],
>>>
>>> [ 'Üres', 'Aláiratlan', 'Aláírásra elküldve', 'Aláírt']
>>> ),
>>> represent = lambda r, row:T(r
>>> )
>>> ),
>>> Field( 'state', 'string',
>>> label = 'Státusz',
>>> requires = IS_IN_SET( ['ok',
>>> 'error'],
>>>
>>> [ 'Rendben', 'Hiba'] ),
>>> represent = lambda r, row:T(r
>>> )
>>> ),
>>> Field( 'data_count', 'integer',
>>> label = 'Adat bejegyzések',
>>> requires = IS_NOT_EMPTY(),
>>> default = 0,
>>> ),
>>> Field( 'dirname', 'string',
>>> label = 'Könyvtár név' ,
>>> requires = IS_NOT_EMPTY(),
>>> ),
>>> Field( 'md5sum', 'string',
>>> label = T( 'MD5 Sum' ),
>>> represent = lambda md5sum,junk
>>> : '[%s]' % md5sum,
>>> writable = False ),
>>> auth.signature,
>>> format = '%(dirname)s/%(name)s (%(id)s)'
>>>  )
>>> db.files_store.created_on.read**able = True
>>> db.define_table( 'file_properties',
>>> Field( 'files_store_id', db.files_store,
>>> label = 'Aláírt fájl' ,
>>> ),
>>> Field( 'kezdo_oldal', 'integer',
>>> label = 'Számla kezdő oldal'
>>> ,
>>> requires = IS_NOT_EMPTY(),
>>> ),
>>> Field( 'oldalszam', 'integer',
>>> label = 'Számla oldalainak
>>> száma',
>>> requires = IS_NOT_EMPTY(),
>>> ),
>>> Field( 'szamla_sorszama', 'string',
>>> label = 'Számla sorszáma',
>>> #requires = IS_NOT_IN_DB(
>>> db, 'file_properties' ),
>>> unique = True,
>>> ),
>>> Field( 'ugyfel_azonosito', 'string',
>>> label = 'Ügyfél azonosító',
>>> requires = IS_NOT_EMPTY(),
>>> ),
>>> Field( 'folyoszamla_azonosito', 'string'
>>> ,
>>> label = 'Folyószámla
>>> azonosító',
>>> requires = IS_NOT_EMPTY(),
>>>

Re: [web2py] Need a little help in code review (a function eating up all memory)

[szimszon]

Should the garbage collector not free up the memory?

2012. május 10., csütörtök 9:28:48 UTC+2 időpontban Bruce Wade a következőt 
írta:
>
> That is how python is. If you want something to clear the memory as soon 
> as you are done with it you need C++ :D
>
> On Thu, May 10, 2012 at 12:27 AM, szimszon  wrote:
>
>> Yes I know but it happens over type the mem usage is linearly growing and 
>> after the successful execution never released and that is why I ask :(
>>
>> 2012. május 10., csütörtök 9:14:14 UTC+2 időpontban Bruce Wade a 
>> következőt írta:
>>
>>> WOW not a good idea: 
>>> for row in db( db.file_**properties.id > 0 ).select(
>>>
>>> If you have a lot of records that is going to kill your memory.
>>>
>>> On Thu, May 10, 2012 at 12:10 AM, szimszon  wrote:
>>>
 I wonder if somebody could help me.

 The following code has eaten up ~1,5GB ram and after ended successfully 
 it is not released :(

 file_properties table has 137314 rows.

 Model:
 db.define_table( 'files_store',
 Field( 'name', 'string',
 label = 'Fájlnév' ,
 ),
 Field( 'content', 'upload',
 uploadseparate = True,
 autodelete = True,
 label = 'Aláírt fájl',
 ),
 Field( 'status', 'string',
 label = 'Állapot',
 requires = IS_IN_SET( [
 'empty', 'notsigned', 'senttosign', 'signed'],
 
 [ 'Üres', 'Aláiratlan', 'Aláírásra elküldve', 'Aláírt'] 
 ),
 represent = lambda r, row:T
 ( r )
 ),
 Field( 'state', 'string',
 label = 'Státusz',
 requires = IS_IN_SET( ['ok'
 , 'error'],
 
 [ 'Rendben', 'Hiba'] ),
 represent = lambda r, row:T
 ( r )
 ),
 Field( 'data_count', 'integer',
 label = 'Adat bejegyzések',
 requires = IS_NOT_EMPTY(),
 default = 0,
 ),
 Field( 'dirname', 'string',
 label = 'Könyvtár név' ,
 requires = IS_NOT_EMPTY(),
 ),
 Field( 'md5sum', 'string',
 label = T( 'MD5 Sum' ),
 represent = lambda md5sum,junk
 : '[%s]' % md5sum,
 writable = False ),
 auth.signature,
 format = '%(dirname)s/%(name)s (%(id)s)'
  )
 db.files_store.created_on.read**able = True
 db.define_table( 'file_properties',
 Field( 'files_store_id', db.files_store
 ,
 label = 'Aláírt fájl' ,
 ),
 Field( 'kezdo_oldal', 'integer',
 label = 'Számla kezdő 
 oldal',
 requires = IS_NOT_EMPTY(),
 ),
 Field( 'oldalszam', 'integer',
 label = 'Számla oldalainak 
 száma',
 requires = IS_NOT_EMPTY(),
 ),
 Field( 'szamla_sorszama', 'string',
 label = 'Számla sorszáma',
 #requires = IS_NOT_IN_DB( 
 db, 'file_properties' ),
 unique = True,
 ),
 Field( 'ugyfel_azonosito', 'string',

Re: [web2py] Need a little help in code review (a function eating up all memory)

[Bruce Wade]

http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object.htm

That is a little more details

On Thu, May 10, 2012 at 12:37 AM, szimszon  wrote:

> Should the garbage collector not free up the memory?
>
> 2012. május 10., csütörtök 9:28:48 UTC+2 időpontban Bruce Wade a
> következőt írta:
>
>> That is how python is. If you want something to clear the memory as soon
>> as you are done with it you need C++ :D
>>
>> On Thu, May 10, 2012 at 12:27 AM, szimszon  wrote:
>>
>>> Yes I know but it happens over type the mem usage is linearly growing
>>> and after the successful execution never released and that is why I ask :(
>>>
>>> 2012. május 10., csütörtök 9:14:14 UTC+2 időpontban Bruce Wade a
>>> következőt írta:
>>>
 WOW not a good idea:
 for row in db( db.file_**propert**ies.id > 0 ).select(

 If you have a lot of records that is going to kill your memory.

 On Thu, May 10, 2012 at 12:10 AM, szimszon  wrote:

> I wonder if somebody could help me.
>
> The following code has eaten up ~1,5GB ram and after ended
> successfully it is not released :(
>
> file_properties table has 137314 rows.
>
> Model:
> db.define_table( 'files_store',
> Field( 'name', 'string',
> label = 'Fájlnév' ,
> ),
> Field( 'content', 'upload',
> uploadseparate = True,
> autodelete = True,
> label = 'Aláírt fájl',
> ),
> Field( 'status', 'string',
> label = 'Állapot',
> requires = IS_IN_SET( [
> 'empty', 'notsigned', 'senttosign', 'signed'],
>
>   [ 'Üres', 'Aláiratlan', 'Aláírásra elküldve',
> 'Aláírt'] ),
> represent = lambda r, row:
> T( r )
> ),
> Field( 'state', 'string',
> label = 'Státusz',
> requires = IS_IN_SET( [
> 'ok', 'error'],
>
>   [ 'Rendben', 'Hiba'] ),
> represent = lambda r, row:
> T( r )
> ),
> Field( 'data_count', 'integer',
> label = 'Adat bejegyzések'
> ,
> requires = IS_NOT_EMPTY(),
> default = 0,
> ),
> Field( 'dirname', 'string',
> label = 'Könyvtár név' ,
> requires = IS_NOT_EMPTY(),
> ),
> Field( 'md5sum', 'string',
> label = T( 'MD5 Sum' ),
> represent = lambda md5sum,junk
> : '[%s]' % md5sum,
> writable = False ),
> auth.signature,
> format = '%(dirname)s/%(name)s (%(id)s)'
>  )
> db.files_store.created_on.readable = True
> db.define_table( 'file_properties',
> Field( 'files_store_id', db.
> files_store,
> label = 'Aláírt fájl' ,
> ),
> Field( 'kezdo_oldal', 'integer',
> label = 'Számla kezdő
> oldal',
> requires = IS_NOT_EMPTY(),
> ),
> Field( 'oldalszam', 'integer',
> label = 'Számla
> oldalainak száma',
> requires = IS_NOT_EMPTY(),
> ),
> Field( 'szamla_sorszama', 'string',
> label = 'Számla sorszáma',
> #requires = IS_NOT_IN_DB(
> db, 'file_properties' ),
> unique = True,
> 

Re: [web2py] Need a little help in code review (a function eating up all memory)

[szimszon]

If I understand well then python release the mem. for reuse by itself but 
not releasing at OS level. Fine.

But then if I rerun the function I'll expect that the mem allocated for 
python is not growing. Because python reuse it.

I executed the controller function and my mem usage at OS level 2x as after 
first run. It sounds not right.

2012. május 10., csütörtök 9:41:31 UTC+2 időpontban Bruce Wade a következőt 
írta:
>
>
> http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object.htm
>
> That is a little more details
>
> On Thu, May 10, 2012 at 12:37 AM, szimszon  wrote:
>
>> Should the garbage collector not free up the memory?
>>
>> 2012. május 10., csütörtök 9:28:48 UTC+2 időpontban Bruce Wade a 
>> következőt írta:
>>
>>> That is how python is. If you want something to clear the memory as soon 
>>> as you are done with it you need C++ :D
>>>
>>> On Thu, May 10, 2012 at 12:27 AM, szimszon  wrote:
>>>
 Yes I know but it happens over type the mem usage is linearly growing 
 and after the successful execution never released and that is why I ask :(

 2012. május 10., csütörtök 9:14:14 UTC+2 időpontban Bruce Wade a 
 következőt írta:

> WOW not a good idea: 
> for row in db( db.file_**propert**ies.id > 0 ).select(
>
> If you have a lot of records that is going to kill your memory.
>
> On Thu, May 10, 2012 at 12:10 AM, szimszon  wrote:
>
>> I wonder if somebody could help me.
>>
>> The following code has eaten up ~1,5GB ram and after ended 
>> successfully it is not released :(
>>
>> file_properties table has 137314 rows.
>>
>> Model:
>> db.define_table( 'files_store',
>> Field( 'name', 'string',
>> label = 'Fájlnév' ,
>> ),
>> Field( 'content', 'upload',
>> uploadseparate = True,
>> autodelete = True,
>> label = 'Aláírt fájl',
>> ),
>> Field( 'status', 'string',
>> label = 'Állapot',
>> requires = IS_IN_SET( [
>> 'empty', 'notsigned', 'senttosign', 'signed'],
>>   
>>   [ 'Üres', 'Aláiratlan', 'Aláírásra elküldve', 
>> 'Aláírt'] ),
>> represent = lambda r, row
>> :T( r )
>> ),
>> Field( 'state', 'string',
>> label = 'Státusz',
>> requires = IS_IN_SET( [
>> 'ok', 'error'],
>>   
>>   [ 'Rendben', 'Hiba'] ),
>> represent = lambda r, row
>> :T( r )
>> ),
>> Field( 'data_count', 'integer',
>> label = 'Adat 
>> bejegyzések',
>> requires = IS_NOT_EMPTY
>> (),
>> default = 0,
>> ),
>> Field( 'dirname', 'string',
>> label = 'Könyvtár név' ,
>> requires = IS_NOT_EMPTY
>> (),
>> ),
>> Field( 'md5sum', 'string',
>> label = T( 'MD5 Sum' ),
>> represent = lambda md5sum
>> , junk: '[%s]' % md5sum,
>> writable = False ),
>> auth.signature,
>> format = '%(dirname)s/%(name)s (%(id)s)'
>>  )
>> db.files_store.created_on.readable = True
>> db.define_table( 'file_properties',
>> Field( 'files_store_id', db.
>> files_store,
>> label = 'Aláírt fájl' ,
>> ),
>> Field( 'kezdo_oldal', 'integer',
>> label = 'Számla kezdő 
>> oldal',
>> requires = IS_NOT_EMPTY
>> (),
>

Re: [web2py] Re: MongoDB Adapter error in select

[Francisco Costa]

I've enumerated fields and colnames and they are like this:

fields:
id
text
integer
string


colnames:
city
age
_id
name


 I believe they don't match and thats why there's this error:


File "/opt/web2py/gluon/dal.py", line 8134, in select
return adapter.select(self.query,fields,attributes)
File "/opt/web2py/gluon/dal.py", line 4711, in select
 return processor(rows,fields,colnames,False)
File "/opt/web2py/gluon/dal.py", line 1681, in parse
self.parse_value(value, fields[j].type,blob_decode)
File "/opt/web2py/gluon/dal.py", line 1575, in parse_value
return self.parsemap[key](value,field


_type)
  File "/opt/web2py/gluon/dal.py", line 1641, in parse_id

return int(value)
ValueError: invalid literal for int() with base 10: 'Toronto'




Francisco Costa
http://franciscocosta.com



On Thu, May 10, 2012 at 7:03 AM, kokoyo  wrote:

> Hi Massimo, i did it and get output (test on user_auth table):
>
> mongoqry_dict ={'email': 'myem...@gmail.com'}
> mongofields_dict = SON([('id':1), ('first_name':1),  ('last_name':
> 1),  ('email':1), ('password':1), ('registration_key':
> 1), ('reset_password_key':1), ('registration_id':1) ])
> mongo_list_dicts = 
>
>
>
> On Thursday, May 10, 2012 1:52:02 PM UTC+9, Massimo Di Pierro wrote:
>>
>> Should be an easy fix but I need your help. Before line 4711 of dal.py
>>
>> return processor(rows,fields,**colnames,False)
>>
>> can you add
>>
>> print rows
>> print fields
>> print colnames
>>
>> what is the output when it fails?
>>
>>
>> On Wednesday, 9 May 2012 05:42:56 UTC-5, Francisco Costa wrote:
>>>
>>> I've enumerated fields and colnames and they are like this:
>>>
>>> fields:
>>> id
>>> text
>>> integer
>>> string
>>>
>>>
>>> colnames:
>>> city
>>> age
>>> _id
>>> name
>>>
>>>
>>>  I believe they don't match and thats why there's this error:
>>>
>>> File "/opt/web2py/gluon/dal.py", line 8134, in select
>>> return adapter.select(self.query,fiel**ds,attributes)
>>> File "/opt/web2py/gluon/dal.py", line 4711, in select
>>>  return processor(rows,fields,colnames**,False)
>>> File "/opt/web2py/gluon/dal.py", line 1681, in parse
>>> self.parse_value(value, fields[j].type,blob_decode)
>>> File "/opt/web2py/gluon/dal.py", line 1575, in parse_value
>>> return self.parsemap[key](value,field
>>>
>>>
>>> _type)
>>>   File "/opt/web2py/gluon/dal.py", line 1641, in parse_id
>>>
>>> return int(value)
>>> ValueError: invalid literal for int() with base 10: 'Toronto'
>>>
>>>
>>>
>>>
>>>
>>> On Monday, May 7, 2012 11:40:36 PM UTC+1, Massimo Di Pierro wrote:

 Please try again.

 On Monday, 7 May 2012 15:48:04 UTC-5, Francisco Costa wrote:
>
> missing a colon at the end of the line 4677
>
> if key == 'id'
>
>
> and now I get this error
>
>   File "/opt/web2py/gluon/dal.py", line 8134, in select
>
> return adapter.select(self.query,fiel**ds,attributes)
>
>   File "/opt/web2py/gluon/dal.py", line 4711, in select
>
> return processor(rows,columns,False)
>
>   File "/opt/web2py/gluon/dal.py", line 1675, in parse
>
> for j,colname in enumerate(colnames):
> TypeError: 'bool' object is not iterable
>
>
> maybe because of the False on line 4711
>
>
>
>
> On Monday, May 7, 2012 9:27:35 PM UTC+1, Massimo Di Pierro wrote:
>>
>> Thanks for your help and check on this. I just committed a fix in
>> trunk that should address it. Please let me know.
>>
>> On Monday, 7 May 2012 15:19:40 UTC-5, Francisco Costa wrote:
>>>
>>> I think that the mongofields_dict.keys() don't match to the data in
>>> the rows
>>>
>>> mongofields_dict:
>>> id
>>> name
>>> age
>>> city
>>> rows:
>>> Toronto
>>> 66L
>>> 24652490551171733682233802752L
>>> John
>>>
>>>
>>> On Monday, May 7, 2012 8:42:33 PM UTC+1, Massimo Di Pierro wrote:

 Your linennumbers do not much the code in trunk. Please try the
 code in trunk. Anyway, something goes wrong in the parse(self, rows,
 fields, colnames, blob_decode=True) function. I suspect mongo does not
 return the columns in the order web2py think it does. Can you print the
 input of the parse function?

 On Monday, 7 May 2012 13:20:39 UTC-5, Francisco Costa wrote:
>
> So I have this:
>
> import sys
> import time
> from gluon.dal import DAL, Field
> mongo = DAL('mongodb://localhost:**27017/tymr')
> mongo.define_table('user',
>  Field('name', 'text'),
>  Field('age',  'integer'),
>  Field('city', 'string')
>  )
>
> def insert_users():
> mongo.user.insert(name='John', age=66, city='Toronto')
> mongo.user.insert(name='Mark', age=43, city='Boston')
> mongo.user.insert(name='Tom',  age=43, city='Detroit')
>  

[web2py] Why does DAL report a syntax error on some executesql queries?

[Johann Spies]

I am working with Postgresql and am trying to get a way of working with 
schemas through web2py.

In psql I can do:

js=# show search_path;
  search_path   

 "$user",public
(1 row)js=# set search_path to toets0;
SET
js=# \dt
 List of relations
 Schema |   Name| Type  | Owner 
+---+---+---
 toets0 | auth_user | table | js
(1 row)

js=# select count(*) from auth_user;
 count 
---
 0
(1 row)




But when I try that in web2py I get syntax errors when I try 'show schema'  
or 'set search_path'.

In [2]: db.executesql('select count(*) from auth_user;')
Out[2]: [(0,)]

In [3]: db.executesql('show schema;')
---
OperationalError  Traceback (most recent call last)
/home/js/web2py/applications/nakb/models/menu.py in ()
> 1 db.executesql('show schema;')

/home/js/web2py/gluon/dal.pyc in executesql(self, query, placeholders,as_dict
)
   6827 self._adapter.execute(query, placeholders)
   6828 else:
-> 6829 self._adapter.execute(query)
   6830 if as_dict:
   6831 if not hasattr(self._adapter.cursor,'description'):

/home/js/web2py/gluon/dal.pyc in execute(self, *a, **b)
   1467 
   1468 def execute(self, *a, **b):
-> 1469 return self.log_execute(*a, **b)
   1470 
   1471 def represent(self, obj, fieldtype):

/home/js/web2py/gluon/dal.pyc in log_execute(self, *a, **b)
   1461 self.db._lastsql = command
   1462 t0 = time.time()
-> 1463 ret = self.cursor.execute(*a, **b)
   1464 self.db._timings.append((command,time.time()-t0))
   1465 del self.db._timings[:-TIMINGSSIZE]

OperationalError: near "show": syntax error

db._lastsql
Out[4]: 'show schema;'


Clearly there is not something wrong with the sql-syntax as it works on 
psql.

Regards
Johann

[web2py] too many redirect message when using login

[man24]

I am new to web2py and has been following the online book to learn it.  I 
was following the 'wiki' example and everything worked fine the first time, 
however, today when I click on any action that requires login or even the 
login link in the menu, web browser cannot open the page and gives 
following message:  'too many redirects occurred trying to open the page'. 
 I am using web2py on a mac.  Thanks for your help.

[web2py] Problem with auth_login: too many redirects

[man24]

Sorry for reposting the question.  I am following the 'wiki' example given 
in the web2py book.  After first implementation, it worked fine, however, 
now when I click on any action that requires login or even clicking the 
'login' from the top menu, I get this response from my web browser:  'too 
many redirects occurred when trying to open'.  Any idea what could be 
wrong.  The 'wiki' app was working fine yesterday and I haven't made any 
changes and now today it's giving this problem.  BTW, I am running web2py 
on a mac.  Thanks for your help.

[web2py] problem related to GAE server

[Prakhar Srivastava]

what is the max image size we can upload through the img tag ?
Because my application is working fine the local server and rock server 
when i upload my application on GAE server then it's not showing properly 
www.pixelofn.appspot.com

this is my application actually it stack because he one photo should appear 
in the middle then navigation come from right hand side.

HOW CAN I SOLVE THIS PROBLEM ???

[web2py] GAE Mail Config.

[Prakhar Srivastava]

Is there anyone config the mail feature for GAE Server

[web2py] GAE

[Prakhar Srivastava]

when i upload my application this waring come up  
2012-05-09 23:02:24.992 
/pixelofn/static/images/album/thumbs/PixelOfNature_fm_03.jpg 404  27ms  0kb 
223.183.189.220 - - [09/May/2012:23:02:24 -0700] "GET 
/pixelofn/static/images/album/thumbs/PixelOfNature_fm_03.jpg HTTP/1.1" 404 
0 "http://www.pixelofn.appspot.com/"; - "www.pixelofn.appspot.com" ms=27 
cpu_ms=0 api_cpu_ms=0 cpm_usd=0.74 
 W 
2012-05-09 23:02:24.992 
Static file referenced by handler not found: 
applications/pixelofn/static/images/album/thumbs/PixelOfNature_fm_03.jpg

[web2py] Sending files to a controller and nicedit in the view.

[Jason Brower]

Is there a way I can build a form on one controller.  Then send data to 
it including a file and various other things

And then save that information to a database?
Sounds simple right?
I want to do it dynamically with my nicEdit integration.  That is, I do 
something just like this...

http://www.rkblog.rk.edu.pl/w/p/niceditor-django/#2
But in Web2py.
I think this is the best approach but I want your opinion:
1. Create a form in a page. get_file for example
2. Create another page that has the nicEdit form.
3. When the nicEdit pushes an image to the site it goes to get_file.

Is this correct? I think I can't do this with default settings as there 
is something stoping CSRF attacks, right?

BR,
Jason

[web2py] Difference between web2py reference book edition4.0 and Web2py application development cookbook

[Amit]

Hi,
Can anyone please elaborate what is the difference between Web2py reference 
book edition 4.0 and Web2py application development cookbook? 
I have been searching some good coding example of implementation of SOAP 
web service using web2py but Couldn't get any good example in Web2py 
reference book in details :(, I dint read Web2py cookbook so can anyone put 
some good points about this book, is it having some good coding examples 
related to SOAP web services OR can anybody suggest some good book which 
covers Web services implementation in web2py ?

Hoping for you people good response.

Regards,
Amit

[web2py] Re: SQLFORM.smartgrid : error when setting fields to be displayed as a fields dictionary.

[François Delpierre]

Houhou !

If works also on the smartgrid :
form = SQLFORM.smartgrid(
db.t_xlsfile,
fields=[
db.t_xlsfile.f_xlsfile,
],
),


smartgrid are really powerful, but still missing some docs... I had some 
difficulties to find also how to get the exact display before I found the 
 singular keyword on the table ;-)

Great,



Thanks,

Le jeudi 10 mai 2012 06:57:00 UTC+2, Massimo Di Pierro a écrit :
>
> Can you try:
>
> form = SQLFORM.grid(
> db.t_xlsfile,
> fields=[
> db.t_xlsfile.f_xlsfile,
> ],
> ),
>
> do you get the same error. Can you try web2py trunk as well?
>
>
> On Wednesday, 9 May 2012 19:08:31 UTC-5, François Delpierre wrote:
>>
>> Hi,
>>
>> When I try to select the fields to be displayed as follow :
>> form = SQLFORM.smartgrid(
>> db.t_xlsfile,
>> fields=dict(
>> t_xlsfile=[
>> db.t_xlsfile.f_xlsfile,
>> ],
>> ),
>>
>> I immediately get this strange error :
>> Traceback (most recent call last):
>>   File "/home/www-data/web2py/gluon/restricted.py", line 205, inrestricted
>> exec ccode in environment
>>   File "/home/www-data/web2py/applications/init/controllers/default.py",line 
>> 425, in 
>>   File "/home/www-data/web2py/gluon/globals.py", line 173, in 
>> self._caller = lambda f: f()
>>   File "/home/www-data/web2py/gluon/tools.py", line 2575, in f
>> return action(*a, **b)
>>   File "/home/www-data/web2py/applications/init/controllers/default.py",line 
>> 257, in posting_history
>> t_xlsfile='Table of Excel files',
>>   File "/home/www-data/web2py/gluon/sqlhtml.py", line 1989, in smartgrid
>> user_signature=user_signature,**kwargs)
>>   File "/home/www-data/web2py/gluon/sqlhtml.py", line 1517, in grid
>> if field._tablename in tablenames]
>> AttributeError: 'str' object has no attribute '_tablename'
>>
>> Any idea ??
>>
>

[web2py] Re: Sending files to a controller and nicedit in the view.

[Anthony]

>
> Is this correct? I think I can't do this with default settings as there 
> is something stoping CSRF attacks, right?
>

If your get_file function creates a web2py form and calls form.process(), 
then by default it will add a _formkey token to the form for CSRF 
protection -- if that key is not present upon submission, the form will not 
be accepted. To prevent that, you can just do form.process(session=None). 
You could also forego the web2py form processing and just handle the upload 
manually (http://web2py.com/books/default/chapter/29/6#Manual-uploads). On 
the other hand, if you still want the CSRF protection, you could call the 
get_file function via Ajax from the main page in order to set and get the 
_formkey token, and then configure nicEdit to post the _formkey value along 
with the file (if it can do that). Another option might be using a 
digitally signed URL for get_file (
http://web2py.com/books/default/chapter/29/4#Digitally-signed-urls).

Anthony

Re: [web2py] Combination of fields must be unique

[Bruno Rocha]

db.define_table('nm', Field('a', notnull=True),  Field('b', notnull=True),
Field('unikey', unique=True))

db.nm.unikey.compute = lambda row: row.a + row.b
Em 10/05/2012 02:46, "Alec Taylor"  escreveu:

> E.g.: For a letter to be unique and identifiable it needs unique(location
> && name)
>
> How do I make a requirement that a combination of fields must be unique?
>
> Thanks for all information,
>
> Alec Taylor
>

[web2py] Re: Difference between web2py reference book edition4.0 and Web2py application development cookbook

[Anthony]

The cookbook is a collection of recipes for implementing specific 
functionality and generally more advanced than the online reference book. 
The cookbook does include one SOAP example, and you can see the sample code 
for it in this app: 
https://github.com/mdipierro/web2py-recipes-source/blob/master/apps/07_Web_services/web2py.app.webservices.w2p
 (all 
the example code from the cookbook is available in this repo: 
https://github.com/mdipierro/web2py-recipes-source). The cookbook (and 
repo) include other web services examples as well. There are also some 
services recipes at web2pyslices.com: 
http://www.web2pyslices.com/article/search?page=2&q=service.

Anthony

On Thursday, May 10, 2012 6:24:08 AM UTC-4, Amit wrote:
>
> Hi,
> Can anyone please elaborate what is the difference between Web2py 
> reference book edition 4.0 and Web2py application development cookbook? 
> I have been searching some good coding example of implementation of SOAP 
> web service using web2py but Couldn't get any good example in Web2py 
> reference book in details :(, I dint read Web2py cookbook so can anyone put 
> some good points about this book, is it having some good coding examples 
> related to SOAP web services OR can anybody suggest some good book which 
> covers Web services implementation in web2py ?
>
> Hoping for you people good response.
>
> Regards,
> Amit
>

[web2py] Re: Difference between web2py reference book edition4.0 and Web2py application development cookbook

[Amit]

Thanks Anthony :), let me check it out if it really helps me implement web 
services in web2py.

On Thursday, 10 May 2012 16:16:55 UTC+5:30, Anthony wrote:
>
> The cookbook is a collection of recipes for implementing specific 
> functionality and generally more advanced than the online reference book. 
> The cookbook does include one SOAP example, and you can see the sample code 
> for it in this app: 
> https://github.com/mdipierro/web2py-recipes-source/blob/master/apps/07_Web_services/web2py.app.webservices.w2p
>  (all 
> the example code from the cookbook is available in this repo: 
> https://github.com/mdipierro/web2py-recipes-source). The cookbook (and 
> repo) include other web services examples as well. There are also some 
> services recipes at web2pyslices.com: 
> http://www.web2pyslices.com/article/search?page=2&q=service.
>
> Anthony
>
> On Thursday, May 10, 2012 6:24:08 AM UTC-4, Amit wrote:
>>
>> Hi,
>> Can anyone please elaborate what is the difference between Web2py 
>> reference book edition 4.0 and Web2py application development cookbook? 
>> I have been searching some good coding example of implementation of SOAP 
>> web service using web2py but Couldn't get any good example in Web2py 
>> reference book in details :(, I dint read Web2py cookbook so can anyone put 
>> some good points about this book, is it having some good coding examples 
>> related to SOAP web services OR can anybody suggest some good book which 
>> covers Web services implementation in web2py ?
>>
>> Hoping for you people good response.
>>
>> Regards,
>> Amit
>>
>

[web2py] Re: Why does DAL report a syntax error on some executesql queries?

[Johann Spies]

Problem partly solved:  there was a mistake in my database connection:  It 
was configured wrongly using sqlite in stead of postgresql.

Now I can do db.executesql('show search_path')  and set the search path but 
I still cannot do: db.executesql('show schema')

Regards
Johann

Re: [web2py] new feature in trunk: better markmin

[Alan Etkin]

That solves it. Even it's possible to populate the input with different 
:html instances. Perhaps could come handy to reserve a tag for html, saving 
the extra lambda argument.

On Wednesday, May 9, 2012 2:03:00 AM UTC-3, Massimo Di Pierro wrote:
>
> You can use MARMIN(...,extra=dict(html=lambda x: x))
>
> then you would embed with """
>
> ``this is html``:html
>
> """
>
> but kind of defies the purpose and you may want to sanitize(x)
>
>

Re: [web2py] Combination of fields must be unique

[Anthony]

If inserts into the table will be done via form submission and you don't 
want to store an extra field in the table, another option is:

db.define_table('letter',
Field('name'),
Field('location',
requires=IS_NOT_IN_DB(db(db.letter.name==request.vars.name), 
'letter.location')))

That will make sure location is unique among the set of records where name 
matches the new value of name being inserted (so the combination of name 
and location must be unique). It depends on the value of name being 
available in request.vars, so as is, this method will only work with 
form-based inserts -- otherwise, you could do:

db.letter.location.requires = IS_NOT_IN_DB(db(db.letter.name==my_new_name), 
'letter.location')
result = db.letter.validate_and_insert(name=my_new_name, location=
my_new_location)
if result.errors:
print result.errors

Another option with form submissions is to use an onvalidation function: 
http://web2py.com/books/default/chapter/29/7#onvalidation.

Anthony

On Thursday, May 10, 2012 6:36:18 AM UTC-4, rochacbruno wrote:
>
> db.define_table('nm', Field('a', notnull=True),  Field('b', notnull=True), 
> Field('unikey', unique=True))
>
> db.nm.unikey.compute = lambda row: row.a + row.b
> Em 10/05/2012 02:46, "Alec Taylor"  escreveu:
>
>> E.g.: For a letter to be unique and identifiable it needs unique(location 
>> && name)
>>
>> How do I make a requirement that a combination of fields must be unique?
>>
>> Thanks for all information,
>>
>> Alec Taylor
>>
>

Re: [web2py] override smartgrid 'add' button

[Franzé Jr]

On Thu, May 10, 2012 at 1:43 AM, Massimo Di Pierro <
massimo.dipie...@gmail.com> wrote:

> Are you trying to remove the header? I do not think we have an option for
> that. You can try hide it using jquery.


Yes..I'm trying... I can use jQuery, but I think it may have a option to do
this thing...
-- 
Franzé Jr
http://lia.ufc.br/~franzejr

[web2py] Re: Looking for web2py help - is there a group for recruiting?

[Anthony]

Also here: http://experts4solutions.com/

On Thursday, May 10, 2012 1:27:31 AM UTC-4, David Rager wrote:
>
> We will be looking for some web2py programming help in June - curious 
> where the best place is to find folks.  Is it here?
>
> Thanks!
>

[web2py] Re: Problem with auth_login: too many redirects

[Anthony]

Can you post your exact code (models, controllers, views)?

On Thursday, May 10, 2012 1:22:14 AM UTC-4, man24 wrote:
>
> Sorry for reposting the question.  I am following the 'wiki' example given 
> in the web2py book.  After first implementation, it worked fine, however, 
> now when I click on any action that requires login or even clicking the 
> 'login' from the top menu, I get this response from my web browser:  'too 
> many redirects occurred when trying to open'.  Any idea what could be 
> wrong.  The 'wiki' app was working fine yesterday and I haven't made any 
> changes and now today it's giving this problem.  BTW, I am running web2py 
> on a mac.  Thanks for your help.

Re: [web2py] Combination of fields must be unique

[Johann Spies]

I let the database (Postgresql) do the work using a constraint and then do
a

try:
 
except:
 pass

in web2py.

Regards
Johann
-- 
Because experiencing your loyal love is better than life itself,
my lips will praise you.  (Psalm 63:3)

Re: [web2py] override smartgrid 'add' button

[Jim Steil]

I would really like to see an option to override the default Add 
button.  I have a couple situations where I'd like to provide sort of a 
dropdown menu button there to give the option to add a couple different 
things.  Kind of like what Outlook offers on the New button.



Better yet would be to offer this within the .grid so I could just 
provide a list of dicts with the label and url and have the .grid figure 
out how to display this.  If only one entry in the list, display as 
button, otherwise provide the dropdown menu button with the options.


Anyone else interested in something like this?

-Jim

On 5/10/2012 6:07 AM, Franzé Jr wrote:



On Thu, May 10, 2012 at 1:43 AM, Massimo Di Pierro 
mailto:massimo.dipie...@gmail.com>> wrote:


Are you trying to remove the header? I do not think we have an
option for that. You can try hide it using jquery.


Yes..I'm trying... I can use jQuery, but I think it may have a option 
to do this thing...

--
Franzé Jr
http://lia.ufc.br/~franzejr 

<>

[web2py] Re: too many redirect message when using login

[Massimo Di Pierro]

Please open a ticket about this. Meanwhile, can you try delete all session 
files? Any progress?

On Wednesday, 9 May 2012 23:27:28 UTC-5, man24 wrote:
>
> I am new to web2py and has been following the online book to learn it.  I 
> was following the 'wiki' example and everything worked fine the first time, 
> however, today when I click on any action that requires login or even the 
> login link in the menu, web browser cannot open the page and gives 
> following message:  'too many redirects occurred trying to open the page'. 
>  I am using web2py on a mac.  Thanks for your help.

Re: [web2py] Need a little help in code review (a function eating up all memory)

[Bruce Wade]

Their is no grantee that python will use that exact same memory. It would
also depend how frequently you use that function.

http://mg.pov.lt/blog/hunting-python-memleaks.html

On Thu, May 10, 2012 at 12:57 AM, szimszon  wrote:

> If I understand well then python release the mem. for reuse by itself but
> not releasing at OS level. Fine.
>
> But then if I rerun the function I'll expect that the mem allocated for
> python is not growing. Because python reuse it.
>
> I executed the controller function and my mem usage at OS level 2x as
> after first run. It sounds not right.
>
> 2012. május 10., csütörtök 9:41:31 UTC+2 időpontban Bruce Wade a
> következőt írta:
>
>> http://effbot.org/pyfaq/why-**doesnt-python-release-the-**
>> memory-when-i-delete-a-large-**object.htm
>>
>> That is a little more details
>>
>> On Thu, May 10, 2012 at 12:37 AM, szimszon  wrote:
>>
>>> Should the garbage collector not free up the memory?
>>>
>>> 2012. május 10., csütörtök 9:28:48 UTC+2 időpontban Bruce Wade a
>>> következőt írta:
>>>
 That is how python is. If you want something to clear the memory as
 soon as you are done with it you need C++ :D

 On Thu, May 10, 2012 at 12:27 AM, szimszon  wrote:

> Yes I know but it happens over type the mem usage is linearly growing
> and after the successful execution never released and that is why I ask :(
>
> 2012. május 10., csütörtök 9:14:14 UTC+2 időpontban Bruce Wade a
> következőt írta:
>
>> WOW not a good idea:
>> for row in db( db.file_**properties.id > 0 ).select(
>>
>> If you have a lot of records that is going to kill your memory.
>>
>> On Thu, May 10, 2012 at 12:10 AM, szimszon wrote:
>>
>>> I wonder if somebody could help me.
>>>
>>> The following code has eaten up ~1,5GB ram and after ended
>>> successfully it is not released :(
>>>
>>> file_properties table has 137314 rows.
>>>
>>> Model:
>>> db.define_table( 'files_store',
>>> Field( 'name', 'string',
>>> label = 'Fájlnév' ,
>>> ),
>>> Field( 'content', 'upload',
>>> uploadseparate = True,
>>> autodelete = True,
>>> label = 'Aláírt fájl',
>>> ),
>>> Field( 'status', 'string',
>>> label = 'Állapot',
>>> requires = IS_IN_SET( [
>>> 'empty', 'notsigned', 'senttosign', 'signed'],
>>>
>>> [ 'Üres', 'Aláiratlan', 'Aláírásra elküldve',
>>> 'Aláírt'] ),
>>> represent = lambda r,row
>>> :T( r )
>>> ),
>>> Field( 'state', 'string',
>>> label = 'Státusz',
>>> requires = IS_IN_SET( [
>>> 'ok', 'error'],
>>>
>>> [ 'Rendben', 'Hiba'] ),
>>> represent = lambda r,row
>>> :T( r )
>>> ),
>>> Field( 'data_count', 'integer',
>>> label = 'Adat
>>> bejegyzések',
>>> requires = IS_NOT_EMPTY
>>> (),
>>> default = 0,
>>> ),
>>> Field( 'dirname', 'string',
>>> label = 'Könyvtár név' ,
>>> requires = IS_NOT_EMPTY
>>> (),
>>> ),
>>> Field( 'md5sum', 'string',
>>> label = T( 'MD5 Sum' ),
>>> represent = lambdamd5sum
>>> , junk: '[%s]' % md5sum,
>>> writable = False ),
>>> auth.signature,
>>> format = '%(dirname)s/%(name)s (%(id)s)'
>>>  )
>>> db.files_store.created_on.read**able = True
>>> db.define_table( 'file_properties',
>>> Field( 'files_store_id', db.
>>> files_store,
>>> label = 'Aláírt fájl' ,
>>>

[web2py] Re: MongoDB Adapter error in select

[Massimo Di Pierro]

Please try trunk again.

On Thursday, 10 May 2012 01:03:22 UTC-5, kokoyo wrote:
>
> Hi Massimo, i did it and get output (test on user_auth table):
>
> mongoqry_dict ={'email': 'myem...@gmail.com'}
> mongofields_dict = SON([('id':1), ('first_name':1),  ('last_name':
> 1),  ('email':1), ('password':1), ('registration_key':
> 1), ('reset_password_key':1), ('registration_id':1) ])
> mongo_list_dicts = 
>
>
>
> On Thursday, May 10, 2012 1:52:02 PM UTC+9, Massimo Di Pierro wrote:
>>
>> Should be an easy fix but I need your help. Before line 4711 of dal.py
>>
>> return processor(rows,fields,colnames,False)
>>
>> can you add
>>
>> print rows
>> print fields
>> print colnames
>>
>> what is the output when it fails?
>>
>>
>> On Wednesday, 9 May 2012 05:42:56 UTC-5, Francisco Costa wrote:
>>>
>>> I've enumerated fields and colnames and they are like this:
>>>
>>> fields:
>>> id
>>> text
>>> integer
>>> string
>>>
>>>
>>> colnames:
>>> city
>>> age
>>> _id
>>> name
>>>
>>>
>>>  I believe they don't match and thats why there's this error:
>>>
>>> File "/opt/web2py/gluon/dal.py", line 8134, in select
>>> return adapter.select(self.query,fields,attributes)
>>> File "/opt/web2py/gluon/dal.py", line 4711, in select
>>> return processor(rows,fields,colnames,False)
>>> File "/opt/web2py/gluon/dal.py", line 1681, in parse
>>> self.parse_value(value, fields[j].type,blob_decode)
>>> File "/opt/web2py/gluon/dal.py", line 1575, in parse_value
>>> return self.parsemap[key](value,field
>>>
>>> _type)
>>>   File "/opt/web2py/gluon/dal.py", line 1641, in parse_id
>>>
>>> return int(value)
>>> ValueError: invalid literal for int() with base 10: 'Toronto'
>>>
>>>
>>>
>>>
>>>
>>> On Monday, May 7, 2012 11:40:36 PM UTC+1, Massimo Di Pierro wrote:

 Please try again.

 On Monday, 7 May 2012 15:48:04 UTC-5, Francisco Costa wrote:
>
> missing a colon at the end of the line 4677
>
> if key == 'id'
>
>
> and now I get this error
>
>   File "/opt/web2py/gluon/dal.py", line 8134, in select
> return adapter.select(self.query,fields,attributes)
>   File "/opt/web2py/gluon/dal.py", line 4711, in select
> return processor(rows,columns,False)
>   File "/opt/web2py/gluon/dal.py", line 1675, in parse
> for j,colname in enumerate(colnames):
> TypeError: 'bool' object is not iterable
>
>
> maybe because of the False on line 4711
>
>
>
>
> On Monday, May 7, 2012 9:27:35 PM UTC+1, Massimo Di Pierro wrote:
>>
>> Thanks for your help and check on this. I just committed a fix in 
>> trunk that should address it. Please let me know.
>>
>> On Monday, 7 May 2012 15:19:40 UTC-5, Francisco Costa wrote:
>>>
>>> I think that the mongofields_dict.keys() don't match to the data in 
>>> the rows
>>>
>>> mongofields_dict:
>>> id
>>> name
>>> age
>>> city
>>> rows:
>>> Toronto
>>> 66L
>>> 24652490551171733682233802752L
>>> John
>>>
>>>
>>> On Monday, May 7, 2012 8:42:33 PM UTC+1, Massimo Di Pierro wrote:

 Your linennumbers do not much the code in trunk. Please try the 
 code in trunk. Anyway, something goes wrong in the parse(self, rows, 
 fields, colnames, blob_decode=True) function. I suspect mongo does not 
 return the columns in the order web2py think it does. Can you print 
 the 
 input of the parse function?

 On Monday, 7 May 2012 13:20:39 UTC-5, Francisco Costa wrote:
>
> So I have this:
>
> import sys
> import time
> from gluon.dal import DAL, Field
> mongo = DAL('mongodb://localhost:27017/tymr')
> mongo.define_table('user',
>  Field('name', 'text'),
>  Field('age',  'integer'),
>  Field('city', 'string')
>  )
>
> def insert_users():
> mongo.user.insert(name='John', age=66, city='Toronto')
> mongo.user.insert(name='Mark', age=43, city='Boston')
> mongo.user.insert(name='Tom',  age=43, city='Detroit')
> mongo.user.insert(name='Jim',  age=18, city='Detroit')
> mongo.user.insert(name='Jack', age=18)
> mongo.user.insert(name='Eric', city='Boston')
> return 'users in database'
>
> def find_users():
> users = mongo(mongo.user.age==66).select()
> return dict(users=users)
>
>
> after I run *insert_users* I check in MongoDb via terminal and 
> everything is correct:
> > db.user.find()
> { "_id" : ObjectId("4fa80feea34feb34f800"), "city" : 
> "Toronto", "age" : NumberLong(66), "name" : "John" }
> { "_id" : ObjectId("4fa80feea34feb34f801"), "city" : "Boston", 
> "age" : NumberLong(43), "name" : "Mark" }
> { "_id" : ObjectId("4f

[web2py] Re: SQLFORM.smartgrid : error when setting fields to be displayed as a fields dictionary.

[Massimo Di Pierro]

Still your original code should have worked. Please submit a bug report.

On Thursday, 10 May 2012 05:26:57 UTC-5, François Delpierre wrote:
>
> Houhou !
>
> If works also on the smartgrid :
> form = SQLFORM.smartgrid(
> db.t_xlsfile,
> fields=[
> db.t_xlsfile.f_xlsfile,
> ],
> ),
>
>
> smartgrid are really powerful, but still missing some docs... I had some 
> difficulties to find also how to get the exact display before I found the 
>  singular keyword on the table ;-)
>
> Great,
>
>
>
> Thanks,
>
> Le jeudi 10 mai 2012 06:57:00 UTC+2, Massimo Di Pierro a écrit :
>>
>> Can you try:
>>
>> form = SQLFORM.grid(
>> db.t_xlsfile,
>> fields=[
>> db.t_xlsfile.f_xlsfile,
>> ],
>> ),
>>
>> do you get the same error. Can you try web2py trunk as well?
>>
>>
>> On Wednesday, 9 May 2012 19:08:31 UTC-5, François Delpierre wrote:
>>>
>>> Hi,
>>>
>>> When I try to select the fields to be displayed as follow :
>>> form = SQLFORM.smartgrid(
>>> db.t_xlsfile,
>>> fields=dict(
>>> t_xlsfile=[
>>> db.t_xlsfile.f_xlsfile,
>>> ],
>>> ),
>>>
>>> I immediately get this strange error :
>>> Traceback (most recent call last):
>>>   File "/home/www-data/web2py/gluon/restricted.py", line 205, inrestricted
>>> exec ccode in environment
>>>   File 
>>> "/home/www-data/web2py/applications/init/controllers/default.py",line 
>>> 425, in 
>>>   File "/home/www-data/web2py/gluon/globals.py", line 173, in 
>>> self._caller = lambda f: f()
>>>   File "/home/www-data/web2py/gluon/tools.py", line 2575, in f
>>> return action(*a, **b)
>>>   File 
>>> "/home/www-data/web2py/applications/init/controllers/default.py",line 
>>> 257, in posting_history
>>> t_xlsfile='Table of Excel files',
>>>   File "/home/www-data/web2py/gluon/sqlhtml.py", line 1989, in smartgrid
>>> user_signature=user_signature,**kwargs)
>>>   File "/home/www-data/web2py/gluon/sqlhtml.py", line 1517, in grid
>>> if field._tablename in tablenames]
>>> AttributeError: 'str' object has no attribute '_tablename'
>>>
>>> Any idea ??
>>>
>>

Re: [web2py] Need a little help in code review (a function eating up all memory)

[Bruce Wade]

In your for loop, every time you iterate the object instance count is
increased. Now if you call that loop again before python garbage collector
has time to release memory then your for loop will cause more instances to
be increased. Python will not release memory to the os for an object until
every instance pointing to that object is deleted. This is what is known as
a memory leak and yes garbage collected language are at times the worst for
this situation if you don't watch how you code.

Now with a little research you will find you can make that for loop load
only 1 instance in memory instead of 1 instance per record in the database.
If you make it load only a single instance your problem will go away. I am
pretty sure this might be covered in the web2py book.

On Thu, May 10, 2012 at 6:14 AM, Bruce Wade  wrote:

> Their is no grantee that python will use that exact same memory. It would
> also depend how frequently you use that function.
>
> http://mg.pov.lt/blog/hunting-python-memleaks.html
>
>
> On Thu, May 10, 2012 at 12:57 AM, szimszon  wrote:
>
>> If I understand well then python release the mem. for reuse by itself but
>> not releasing at OS level. Fine.
>>
>> But then if I rerun the function I'll expect that the mem allocated for
>> python is not growing. Because python reuse it.
>>
>> I executed the controller function and my mem usage at OS level 2x as
>> after first run. It sounds not right.
>>
>> 2012. május 10., csütörtök 9:41:31 UTC+2 időpontban Bruce Wade a
>> következőt írta:
>>
>>> http://effbot.org/pyfaq/why-**doesnt-python-release-the-**
>>> memory-when-i-delete-a-large-**object.htm
>>>
>>> That is a little more details
>>>
>>> On Thu, May 10, 2012 at 12:37 AM, szimszon  wrote:
>>>
 Should the garbage collector not free up the memory?

 2012. május 10., csütörtök 9:28:48 UTC+2 időpontban Bruce Wade a
 következőt írta:

> That is how python is. If you want something to clear the memory as
> soon as you are done with it you need C++ :D
>
> On Thu, May 10, 2012 at 12:27 AM, szimszon  wrote:
>
>> Yes I know but it happens over type the mem usage is linearly growing
>> and after the successful execution never released and that is why I ask 
>> :(
>>
>> 2012. május 10., csütörtök 9:14:14 UTC+2 időpontban Bruce Wade a
>> következőt írta:
>>
>>> WOW not a good idea:
>>> for row in db( db.file_**properties.id > 0 ).select(
>>>
>>> If you have a lot of records that is going to kill your memory.
>>>
>>> On Thu, May 10, 2012 at 12:10 AM, szimszon wrote:
>>>
 I wonder if somebody could help me.

 The following code has eaten up ~1,5GB ram and after ended
 successfully it is not released :(

 file_properties table has 137314 rows.

 Model:
 db.define_table( 'files_store',
 Field( 'name', 'string',
 label = 'Fájlnév' ,
 ),
 Field( 'content', 'upload',
 uploadseparate = True,
 autodelete = True,
 label = 'Aláírt fájl',
 ),
 Field( 'status', 'string',
 label = 'Állapot',
 requires = IS_IN_SET( [
 'empty', 'notsigned', 'senttosign', 'signed'],

 [ 'Üres', 'Aláiratlan', 'Aláírásra elküldve',
 'Aláírt'] ),
 represent = lambda r,row
 :T( r )
 ),
 Field( 'state', 'string',
 label = 'Státusz',
 requires = IS_IN_SET( [
 'ok', 'error'],

 [ 'Rendben', 'Hiba'] ),
 represent = lambda r,row
 :T( r )
 ),
 Field( 'data_count', 'integer',
 label = 'Adat
 bejegyzések',
 requires = IS_NOT_EMPTY
 (),
 default = 0,
 ),
 Field( 'dirname', 'string',
   

[web2py] jqgrid assistance

[Larry Wapnitsky]

I'm trying out jqgrid (yes, I finally got plugin_wiki working ;)), but 
can't seem to get it to display data.  Here is my code:

*{{=plugin_wiki.widget('jqgrid', db.ips)}}*

I see a table with MOST of my fields showing (not all, strangely), but no 
data.  I know for a fact that the database is populated.

Ideas?

Thanks

[web2py] Re: MongoDB Adapter error in select

[Francisco Costa]

Still not working. I think Fields order is wrong.

At the end of the select() function I write this:

a = []
for f in fields:
a.append(f.type)
return dict(rows=rows, fields=a, colnames=colnames)

and I got this:

colnames:
city
age
_id
name
fields:
id
text
integer
string
rows:
Toronto
66L
24652490551171733682233802752L
John

Re: [web2py] Need a little help in code review (a function eating up all memory)

[Bruno Rocha]

Just for curiosity, what happens if you do it in pure sql?

for row in db.executesql("select * from file_properties where id > 0"):

# do something


Does it have a lower memory usage?


On Thu, May 10, 2012 at 4:14 AM, Bruce Wade  wrote:

> for row in db( db.file_properties.id > 0 ).select(




-- 

Bruno Rocha
[http://rochacbruno.com.br]

[web2py] Re: Nginx-uwsgi problem.

[Bruce Wade]

Hi,

Have you ever fixed this problem? I am finding it very very hard to handle 
high load websites, also using postgres. I think the bottle neck is the DB 
layer as even using the admin tool causes timeout issues. Last night I had 
2.3k concurrent users for 5 hours which caused one of the servers CPU to 
hit 175% (VPN Node) and another to hit 235%. I am going to completely 
rewrite all my models into modules to see if it helps. However if that 
doesn't work I am not sure what else to try.

--
Bruce

On Monday, September 19, 2011 9:00:12 PM UTC-7, pbreit wrote:
>
> "Isn't it a good idea to send a verification email to prevent fake 
> emails and a 'captcha' to prevent automated registrations !"
>
> No and no. Both of those are customer-hostile and usually unnecessary.
>

Re: [web2py] Need a little help in code review (a function eating up all memory)

[szimszon]

I reduced the code in controller to:

def autoadjust():
lista = list()
last_row = None
next_page_number = 0
for row in db.executesql( "select * from file_properties where id > 0" 
):
pass
lista = TABLE( *lista )
return dict( lista = lista )


And I still have memleak :(

I start web2py:

USER   PID %CPU %MEMVSZ   RSS TTY  STAT START   TIME COMMAND
1008017340  4.6  0.3 156896 15372 pts/9Sl+  16:19   0:00 python ./
web2py.py

After first run:

USER   PID %CPU %MEMVSZ   RSS TTY  STAT START   TIME COMMAND
1008017340  0.7  2.1 237932 87304 pts/9Sl+  16:19   0:00 python ./
web2py.py

2th run:
USER   PID %CPU %MEMVSZ   RSS TTY  STAT START   TIME COMMAND
1008017340  0.8  3.7 302328 151600 pts/9   Sl+  16:19   0:01 python ./
web2py.py

and so on... I'm a bit puzzled. I'm not an expert regarding python mem 
problems :( So please be patient with me.


2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a 
következőt írta:
>
> Just for curiosity, what happens if you do it in pure sql?
>
> for row in db.executesql("select * from file_properties where id > 0"):
>
> # do something
>
>
> Does it have a lower memory usage?
>
>
> On Thu, May 10, 2012 at 4:14 AM, Bruce Wade <> wrote:
>
>> for row in db( db.file_properties.id > 0 ).select(
>
>
>
>
> -- 
>
> Bruno Rocha
> [http://rochacbruno.com.br]
>
>

Re: [web2py] Need a little help in code review (a function eating up all memory)

[szimszon]

It's postgres:// in a 
Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and

Python 2.7.3 (default, Apr 20 2012, 22:44:07) 
[GCC 4.6.3] on linux2

python-psycopg2 2.4.5-1

2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a 
következőt írta:
>
> Just for curiosity, what happens if you do it in pure sql?
>
> for row in db.executesql("select * from file_properties where id > 0"):
>
> # do something
>
>
> Does it have a lower memory usage?
>
>
> On Thu, May 10, 2012 at 4:14 AM, Bruce Wade < >wrote:
>
>> for row in db( db.file_properties.id > 0 ).select(
>
>
>
>
> -- 
>
> Bruno Rocha
> [http://rochacbruno.com.br]
>
>

Re: [web2py] Need a little help in code review (a function eating up all memory)

[Bruce Wade]

Using direct sql query or DAL is going to cause the exact same problem in
this situation.

On Thu, May 10, 2012 at 7:28 AM, szimszon  wrote:

> It's postgres:// in a
> Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and
>
> Python 2.7.3 (default, Apr 20 2012, 22:44:07)
> [GCC 4.6.3] on linux2
>
> python-psycopg2 2.4.5-1
>
>
> 2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a
> következőt írta:
>
>> Just for curiosity, what happens if you do it in pure sql?
>>
>>
>> for row in db.executesql("**select * from file_properties where id > 0"):
>>
>> # do something
>>
>>
>> Does it have a lower memory usage?
>>
>>
>> On Thu, May 10, 2012 at 4:14 AM, Bruce Wade < >wrote:
>>
>>> for row in db( db.file_**properties.id > 0 ).select(
>>
>>
>>
>>
>> --
>>
>> Bruno Rocha
>> [http://rochacbruno.com.br]
>>
>>


-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

[web2py] call for ideas

[Niphlod]

Hi all,
I'm in the process of releasing an app written in web2py that use 
extensively the scheduler. It's going to be used on desktops, but I'd like 
to keep my mind open and if someone will want to load it into 
apache/nginx/uswgi, why not ?

For the intrinsic nature of the app, some logic is completely decoupled 
from the webserver and put in the scheduler, with the web app actually 
scheduling and polling with ajax for results.

As soon as I figure out the license type, run a major cleanup and learn to 
use hg-git to test github, I'll release the app to the public, and I'll ask 
a ton of questions here :P

I had a design issue for "cron execution" of a scheduler, that was "easily" 
fixed with the -E option of gluon/scheduler.py 
Now, another one faces up: would it be possible to start/stop the scheduler 
process from within web2py ? 
I'm familiar with the idea that spawning processes without waiting for them 
is not a good call on a web app, nor webapps are good candidates for 
managing states like "terminating myself" but nonetheless I'll ask here for 
your ideas.

I don't mind having the scheduler always active 24/7 but it would be nice 
for a desktop app to be able to terminate the scheduler when it exits (or 
in the case of "open minded" case, having a disposable scheduler started on 
purpose, that will die on itself when no tasks are found).

What I'd like to do is something like :

- having the scheduler set with "max_empty_runs" to a nice amount of 20 or 
so
- have a method to start it (obviously I'll add some tasks to do before)
- let it run the tasks and then die if there aren't new tasks to do

The scheduler is a nice piece of code, and it was coded with a different 
intent: have a process (or several of them) always active and eager to do 
tasks for "typycal" webapplications run on servers. 

[web2py] Re: Nginx-uwsgi problem.

[Ross Peoples]

The DB layer is usually the bottleneck. However, moving from models to 
modules should reduce any bottleneck caused by the web server.

Re: [web2py] Need a little help in code review (a function eating up all memory)

[szimszon]

Sorry for my dumbness but if something is wrong with my code please point 
me the right line. I'm not so good in English if it comes to "object 
instance count" and so. Yeah I know I should go and do some milkmaid job :) 
but I'm curious.

I'm just define some variable:
lista = list()
last_row = None
next_page_number = 0

Go in a for loop that just assign the db query result one-by-one to row 
variable.

After that I assign the TABLE() helper to a list variable. That is it.



2012. május 10., csütörtök 16:34:49 UTC+2 időpontban Bruce Wade a 
következőt írta:
>
> Using direct sql query or DAL is going to cause the exact same problem in 
> this situation.
>
> On Thu, May 10, 2012 at 7:28 AM, szimszon  wrote:
>
>> It's postgres:// in a 
>> Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and
>>
>> Python 2.7.3 (default, Apr 20 2012, 22:44:07) 
>> [GCC 4.6.3] on linux2
>>
>> python-psycopg2 2.4.5-1
>>
>>
>> 2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a 
>> következőt írta:
>>
>>> Just for curiosity, what happens if you do it in pure sql?
>>>
>>>
>>> for row in db.executesql("**select * from file_properties where id > 
>>> 0"):
>>>
>>> # do something
>>>
>>>
>>> Does it have a lower memory usage?
>>>
>>>
>>> On Thu, May 10, 2012 at 4:14 AM, Bruce Wade < >wrote:
>>>
 for row in db( db.file_**properties.id > 0 ).select(
>>>
>>>
>>>
>>>
>>> -- 
>>>
>>> Bruno Rocha
>>> [http://rochacbruno.com.br]
>>>
>>>
>
>
> -- 
> -- 
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>

Re: [web2py] Re: Nginx-uwsgi problem.

[Michele Comitini]

Is the high load caused by web2py process or postgresql?
Is web2py on the same machine as postgresql?

Use top or htop to find which process uses the CPU.  If it is web2py
*use* the profiler. If it postgres try to find the bad query and try
to improve it by using EXPLAIN.
Could be also the swapping causing the troubles.  On VPS nodes
remember to lower the swappines (see sysctl) to n < 20 or remove swap
completely.


mic


2012/5/10 Ross Peoples :
> The DB layer is usually the bottleneck. However, moving from models to
> modules should reduce any bottleneck caused by the web server.

[web2py] Re: Problem with auth_login: too many redirects

[man24]

Here is the model file - > db2.py 

# coding: utf8
db.define_table('page',
Field('title', requires=IS_NOT_IN_DB(db, 'page.title')),
Field('body', 'text', requires=IS_NOT_EMPTY()),
Field('created_on', 'datetime', default=request.now, 
readable=False, writable=False),
Field('created_by', db.auth_user, default=auth.user_id, 
readable=False, writable=False),
format='%(title)s')

db.define_table('comment',
Field('page_id', db.page, readable=False, writable=False),
Field('body', 'text', requires=IS_NOT_EMPTY()),
Field('created_on', 'datetime', default=request.now, 
readable=False, writable=False),
Field('created_by', db.auth_user, 
default=auth.user_id,readable=False, writable=False))

db.define_table('document',
Field('page_id', db.page, readable=False, writable=False),
Field('name', requires=IS_NOT_IN_DB(db, 'document.name')),
Field('file', 'upload'),
Field('created_on', 'datetime', default=request.now, 
readable=False, writable=False),
Field('created_by', db.auth_user, default=auth.user_id, 
readable=False, writable=False),
format='%(name)s')

Here is the controller: 

def index():
pages = db().select(db.page.id, db.page.title, orderby=db.page.title)
return dict(pages=pages)

@auth.requires_login()
def create():
form=crud.create(db.page, next=URL('index'))
return dict(form=form)

def show():
this_page = db.page(request.args(0)) or redirect(URL('index'))
db.comment.page_id.default = this_page.id
form = crud.create(db.comment) if auth.user else None
pagecomments = db(db.comment.page_id==this_page.id).select()
return dict(page=this_page, comments=pagecomments, form=form)

@auth.requires_login()
def edit():
this_page = db.page(request.args(0)) or redirect(URL('index'))
form = crud.update(db.page, this_page, next=URL('show', 
args=request.args))
return dict(form=form)

@auth.requires_login()
def documents():
page = db.page(request.args(0)) or redirect(URL('index'))
db.document.page_id.default = page.id
db.document.page_id.writable = False
grid = SQLFORM.grid(db.document.page_id==page.id, args=[page.id])
return dict(page=page, grid=grid)

@auth.requires_login()
def user():
"""
exposes:
http:///[app]/default/user/login
http:///[app]/default/user/logout
http:///[app]/default/user/register
http:///[app]/default/user/profile
http:///[app]/default/user/retrieve_password
http:///[app]/default/user/change_password
use @auth.requires_login()
@auth.requires_membership('group name')
@auth.requires_permission('read','table name',record_id)
to decorate functions that need access control
"""
return dict(form=auth())

Here are the views:
Index:
{{left_sidebar_enabled,right_sidebar_enabled=False,True}}
{{extend 'layout.html'}}
Available Wiki Pages
[ {{=A('search', _href=URL('search'))}} ]
{{for page in pages:}}
   {{=LI(A(page.title, _href=URL('show', args=page.id)))}}
{{pass}}
[ {{=A('create page', _href=URL('create'))}}]

Create:
{{extend 'layout.html'}}
Create new wiki page
{{=form}}

Show:
{{extend 'layout.html'}}
{{=page.title}}
[ {{=A('edit', _href=URL('edit', args=request.args))}} ]
[ {{=A('documents', _href=URL('documents', args=request.args))}} ]
{{=MARKMIN(page.body)}}
Comments
{{for comment in comments:}}
  {{=db.auth_user[comment.created_by].first_name}} on 
{{=comment.created_on}} says {{=comment.body}}
{{pass}}
Post a comment
{{=form}}

Edit:
{{extend 'layout.html'}}
Edit wiki page
[ {{=A('show', _href=URL('show', args=request.args))}} ] 
{{=form}}

documents:
{{extend 'layout.html'}}
Documents for page: {{=page.title}}
[ {{=A('show', _href=URL('show', args=request.args))}} ]
Doucments
{{=grid}}

Thanks


On Thursday, May 10, 2012 6:09:09 AM UTC-5, Anthony wrote:
>
> Can you post your exact code (models, controllers, views)?
>
> On Thursday, May 10, 2012 1:22:14 AM UTC-4, man24 wrote:
>>
>> Sorry for reposting the question.  I am following the 'wiki' example 
>> given in the web2py book.  After first implementation, it worked fine, 
>> however, now when I click on any action that requires login or even 
>> clicking the 'login' from the top menu, I get this response from my web 
>> browser:  'too many redirects occurred when trying to open'.  Any idea what 
>> could be wrong.  The 'wiki' app was working fine yesterday and I haven't 
>> made any changes and now today it's giving this problem.  BTW, I am running 
>> web2py on a mac.  Thanks for your help.
>
>

[web2py] Re: Problems trying to use jqgrid from plugin_wiki

[Larry Wapnitsky]

Massimo-

I'm still getting this in the latest trunk of web2py.  Any ideas?

Thanks,
Larry

On Monday, January 10, 2011 4:13:12 PM UTC-5, Massimo Di Pierro wrote:
>
> It took me a while to debug this. 
> The problem is that the ajax service that fills the table requires 
> login. 
> You probably have not created an account or have not logged in. 
> The page that displays the jqgrid does not require login so it shows, 
> but the service responds with a login page instead of sending the 
> data. 
>
> There should be a clearly displayed error in this case and I will 
> modify plugin_wiki to take care of this case more gracefully. 
>
> On Jan 10, 2:00 pm, Lisandro  wrote: 
> > I downloaded web2py (1.91.6) and executed the development server. 
> > I edited the "welcome" app: 
> > 
> > First, I installed the plugin "plugin_wiki" in the "welcome" app. 
> > 
> > Second, I edited the db.py, adding the following line (every other 
> > thing stays as default): 
> > 
> --- 
>  
> > db.define_table('ciudadanos', Field('nro_documento')) 
> > 
> --- 
>  
> > I added some sample records. 
> > 
> > Third, I edited the "default/index.html" view, adding the following 
> > line: 
> > 
> --- 
>  
> > {{=plugin_wiki.widget('jqgrid', 'ciudadanos')}} 
> > 
> --- 
>  
> > 
> > When I acceshttp://.../welcome/default/index, I can see the jqgrid, 
> > but it contains nothing. 
> > I tried with: 
> >  * {{=plugin_wiki.widget('jqgrid',  table='ciudadanos')}} 
> >  * {{=plugin_wiki.widget('jqgrid', table=db.ciudadanos)}} 
> > But nothing happens. 
> > 
> > ¿What am I doing wrong? Thanks in advance.

Re: [web2py] Looking for web2py help - is there a group for recruiting?

[David Rager]

Thanks for the replies!

We have a project building a solar monitoring solution to be used by an O&M 
group which is going to be installed for about 10 different solar plants 
around the world - probably expanded past that next year.  

It is a system that ingests data from multiple devices on a solar plant 
then translates that information into reports and tickets for action by the 
O&M group.  We are building in an analytics layer to enhance insights into 
and efficiency of each solar plant. 

[web2py] Re: too many redirect message when using login

[man24]

Yes, I will open a ticket.  Is there a simpler way of deleting sessions 
that does not involve command line action and is automated?  I am on mac 
and using the compiled web2py.

Also, by mistake, I have posted this question twice  -'Problem with 
auth_login: too many redirects' - so there are now two discussion threads. 
 I apologize, I am new to this group.  I have just pasted the code for my 
model, controller and views in that thread, which is almost verbatim from 
the web2py book.

Thanks

On Thursday, May 10, 2012 8:12:00 AM UTC-5, Massimo Di Pierro wrote:
>
> Please open a ticket about this. Meanwhile, can you try delete all session 
> files? Any progress?
>
> On Wednesday, 9 May 2012 23:27:28 UTC-5, man24 wrote:
>>
>> I am new to web2py and has been following the online book to learn it.  I 
>> was following the 'wiki' example and everything worked fine the first time, 
>> however, today when I click on any action that requires login or even the 
>> login link in the menu, web browser cannot open the page and gives 
>> following message:  'too many redirects occurred trying to open the page'. 
>>  I am using web2py on a mac.  Thanks for your help.
>
>

Re: [web2py] Re: Reloading modules

[howesc]

i use the persistent local datastore on GAE SDK as well.  i have not tried 
to turn on dynamic module re-loading on GAE, but i have a shell script that 
starts/stops GAE and i just restart that.  it's not as fast as 
auto-reloading but it has been ok for me (though now that i'm doing more 
and more in modules it does slow things down)

On Wednesday, May 9, 2012 2:27:33 PM UTC-7, Alex Benfica wrote:
>
> Oi Ricardo, 
>
> Thanks. 
>
> I add the option to use sql lite and databse keeps data when I restart 
> app engine SDK. 
> And about the module reloading... did you have any problems with it? 
>
> On Wed, May 9, 2012 at 5:54 PM, Ricardo Pedroso  
> wrote: 
> > On Wed, May 9, 2012 at 8:35 PM, Alex Benfica  
> wrote: 
> >> 
> >> I'm having a seriuos problem having to stop and start Google App Engine 
> SDK 
> >> when I want edit modules, otherwise, they are not reloaded. 
> >> The problem is that, doing this, I lose the entire database content on 
> GAE 
> >> SDK... 
> > 
> > You can persist the data on GAE SDK 
> > 
> > 
> https://developers.google.com/appengine/docs/python/tools/devserver#Using_the_Datastore
>  
> > 
> > Ricardo 
>
>
>
> -- 
> -- 
>
> Atenciosamente, 
> Alex BENFICA 
>
> -- 
> "O que não se mede não se gerencia." 
>

Re: [web2py] Re: Reloading modules

[Alex BENFICA]

Hi...

When you say "now that i'm doing more and more in modules it does slow
things down"... are you talking about development time, right? is not
about the performace of you web2py app... is that?

The performance increase when using modules instead put code on models
worth all this extra work?
Do you have any measure about this performance gain?


On Thu, May 10, 2012 at 1:17 PM, howesc  wrote:
> i use the persistent local datastore on GAE SDK as well.  i have not tried
> to turn on dynamic module re-loading on GAE, but i have a shell script that
> starts/stops GAE and i just restart that.  it's not as fast as
> auto-reloading but it has been ok for me (though now that i'm doing more and
> more in modules it does slow things down)
>
>
> On Wednesday, May 9, 2012 2:27:33 PM UTC-7, Alex Benfica wrote:
>>
>> Oi Ricardo,
>>
>> Thanks.
>>
>> I add the option to use sql lite and databse keeps data when I restart
>> app engine SDK.
>> And about the module reloading... did you have any problems with it?
>>
>> On Wed, May 9, 2012 at 5:54 PM, Ricardo Pedroso 
>> wrote:
>> > On Wed, May 9, 2012 at 8:35 PM, Alex Benfica 
>> > wrote:
>> >>
>> >> I'm having a seriuos problem having to stop and start Google App Engine
>> >> SDK
>> >> when I want edit modules, otherwise, they are not reloaded.
>> >> The problem is that, doing this, I lose the entire database content on
>> >> GAE
>> >> SDK...
>> >
>> > You can persist the data on GAE SDK
>> >
>> >
>> > https://developers.google.com/appengine/docs/python/tools/devserver#Using_the_Datastore
>> >
>> > Ricardo
>>
>>
>>
>> --
>> --
>>
>> Atenciosamente,
>> Alex BENFICA
>>
>> --
>> "O que não se mede não se gerencia."



-- 
--

Atenciosamente,
Alex BENFICA

--
"O que não se mede não se gerencia."

Re: [web2py] Re: Reloading modules

[Christian Foster Howes]

yes, sorry, i meant in development time.

in one project i moved models to modules and shaved about 33% off of the 
GAE production request time.  another project is in test this week and i 
will hopefully have numbers soon.  note that when you don't need auth 
(not all of my requests use auth) not setting it up saves about 200ms on 
GAE.


cfh

On 5/10/12 9:21 , Alex BENFICA wrote:

Hi...

When you say "now that i'm doing more and more in modules it does slow
things down"... are you talking about development time, right? is not
about the performace of you web2py app... is that?

The performance increase when using modules instead put code on models
worth all this extra work?
Do you have any measure about this performance gain?


On Thu, May 10, 2012 at 1:17 PM, howesc  wrote:

i use the persistent local datastore on GAE SDK as well.  i have not tried
to turn on dynamic module re-loading on GAE, but i have a shell script that
starts/stops GAE and i just restart that.  it's not as fast as
auto-reloading but it has been ok for me (though now that i'm doing more and
more in modules it does slow things down)


On Wednesday, May 9, 2012 2:27:33 PM UTC-7, Alex Benfica wrote:


Oi Ricardo,

Thanks.

I add the option to use sql lite and databse keeps data when I restart
app engine SDK.
And about the module reloading... did you have any problems with it?

On Wed, May 9, 2012 at 5:54 PM, Ricardo Pedroso
wrote:

On Wed, May 9, 2012 at 8:35 PM, Alex Benfica
wrote:


I'm having a seriuos problem having to stop and start Google App Engine
SDK
when I want edit modules, otherwise, they are not reloaded.
The problem is that, doing this, I lose the entire database content on
GAE
SDK...


You can persist the data on GAE SDK


https://developers.google.com/appengine/docs/python/tools/devserver#Using_the_Datastore

Ricardo




--
--

Atenciosamente,
Alex BENFICA

--
"O que não se mede não se gerencia."

Re: [web2py] Need a little help in code review (a function eating up all memory)

[Richard Vézina]

You can't manage what you want to achieve with join?

Richard

On Thu, May 10, 2012 at 10:48 AM, szimszon  wrote:

> Sorry for my dumbness but if something is wrong with my code please point
> me the right line. I'm not so good in English if it comes to "object
> instance count" and so. Yeah I know I should go and do some milkmaid job :)
> but I'm curious.
>
> I'm just define some variable:
>
> lista = list()
> last_row = None
> next_page_number = 0
>
> Go in a for loop that just assign the db query result one-by-one to row
> variable.
>
> After that I assign the TABLE() helper to a list variable. That is it.
>
>
>
> 2012. május 10., csütörtök 16:34:49 UTC+2 időpontban Bruce Wade a
> következőt írta:
>
>> Using direct sql query or DAL is going to cause the exact same problem in
>> this situation.
>>
>> On Thu, May 10, 2012 at 7:28 AM, szimszon  wrote:
>>
>>> It's postgres:// in a
>>> Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and
>>>
>>> Python 2.7.3 (default, Apr 20 2012, 22:44:07)
>>> [GCC 4.6.3] on linux2
>>>
>>> python-psycopg2 2.4.5-1
>>>
>>>
>>> 2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a
>>> következőt írta:
>>>
 Just for curiosity, what happens if you do it in pure sql?


 for row in db.executesql("**sele**ct * from file_properties where id >
 0"):

 # do something


 Does it have a lower memory usage?


 On Thu, May 10, 2012 at 4:14 AM, Bruce Wade < >wrote:

> for row in db( db.file_**propert**ies.id > 0 ).select(




 --

 Bruno Rocha
 [http://rochacbruno.com.br]


>>
>>
>> --
>> --
>> Regards,
>> Bruce Wade
>> http://ca.linkedin.com/in/**brucelwade
>> http://www.wadecybertech.com
>> http://www.fittraineronline.**com  -
>> Fitness Personal Trainers Online
>> http://www.warplydesigned.com
>>
>>

[web2py] Looking for a way to combine auto-complete with adding a non-existing item.

[Remco K]

Hello everyone,

I'm looking for a way to get an auto-complete field combined with adding 
non-existing item to the DB. I've seen a website which
uses this functionality (demo: http://www.tellmehow.nl/video.html )

I have already tried SELECT_OR_ADD_OPTION but i don't like drop-downs...

Thanks in advance!
Remco

[web2py] External database query for authentication (RBAC)

[BoleroDan]

We're looking to use Web2py as our framework for developing a couple 
applications. However we already have a massive database for our research 
project, and almost all of the information is integrated into this database 
(including usernames, passwords, group names etc) so it would be nice to 
have web2py just access this database for username/password/group 
membership authentication.

I'm hoping this can be integrated into Web2Pys already existing RBAC 
management as that would be nice. I also noticed that a lot of the other 
alternative login methods still require web2pys local auth_user table and 
stores the remote credentials locally in this table. Is this needed by 
design or can this be skipped? I'm just thinking its not needed for us to 
have duplicate login information across two databases. I just need Web2py 
to query our database, get the credentials authenticated and what group 
they are in (so i can control access to functions on web2py based on their 
group).

Thanks for any heads up on the matter.

[web2py] why so many instances of console-kit-daemon?

[Carlos]

Hi,

My web2py app runs on ubuntu server 10.04, with postgresql, nginx, 
uwsgi-python.

With htop, I notice there are 64 instances of the following (each with MEM 
0.3%):

   console-kit-daemon --no-daemon

Does anybody know what's that about?.

Does this mean they are using almost 20% of memory (0.3% x 64 instances)?.

Thanks,

   Carlos

[web2py] Re: Nginx-uwsgi problem.

[pbreit]

Is your traffic from serving ads or users coming to your web site? Have you 
exhausted caching opportunities?

Re: [web2py] Need a little help in code review (a function eating up all memory)

[szimszon]

Sorry I don't understand. What do you mean "achieve with join"?

There is an empty for loop with db.executesql() without join. And it is 
eating up the memory. :(

2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a következőt 
írta:
>
> You can't manage what you want to achieve with join?
>
> Richard
>
> On Thu, May 10, 2012 at 10:48 AM, szimszon  wrote:
>
>> Sorry for my dumbness but if something is wrong with my code please point 
>> me the right line. I'm not so good in English if it comes to "object 
>> instance count" and so. Yeah I know I should go and do some milkmaid job :) 
>> but I'm curious.
>>
>> I'm just define some variable:
>>
>> lista = list()
>> last_row = None
>> next_page_number = 0
>>
>> Go in a for loop that just assign the db query result one-by-one to row 
>> variable.
>>
>> After that I assign the TABLE() helper to a list variable. That is it.
>>
>>
>>
>> 2012. május 10., csütörtök 16:34:49 UTC+2 időpontban Bruce Wade a 
>> következőt írta:
>>
>>> Using direct sql query or DAL is going to cause the exact same problem 
>>> in this situation.
>>>
>>> On Thu, May 10, 2012 at 7:28 AM, szimszon  wrote:
>>>
 It's postgres:// in a 
 Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and

 Python 2.7.3 (default, Apr 20 2012, 22:44:07) 
 [GCC 4.6.3] on linux2

 python-psycopg2 2.4.5-1


 2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a 
 következőt írta:

> Just for curiosity, what happens if you do it in pure sql?
>
>
> for row in db.executesql("**sele**ct * from file_properties where id 
> > 0"):
>
> # do something
>
>
> Does it have a lower memory usage?
>
>
> On Thu, May 10, 2012 at 4:14 AM, Bruce Wade < 
> >wrote:
>
>> for row in db( db.file_**propert**ies.id > 0 ).select(
>
>
>
>
> -- 
>
> Bruno Rocha
> [http://rochacbruno.com.br]
>
>
>>>
>>>
>>> -- 
>>> -- 
>>> Regards,
>>> Bruce Wade
>>> http://ca.linkedin.com/in/**brucelwade
>>> http://www.wadecybertech.com
>>> http://www.fittraineronline.**com  - 
>>> Fitness Personal Trainers Online
>>> http://www.warplydesigned.com
>>>
>>>
>

Re: [web2py] Re: Nginx-uwsgi problem.

[Bruce Wade]

Web2py is on 3 different servers/nodes, postgresql is on it's own node with
8GB ram.

CPU is being used by uwsgi so web2py. The slowness I think is from DB
queries as when you load a page without the DB involved much it loads
quickly

The serving ads part is not a problem it is the other pages on the website.
At least not the adviewer the banner ads are new. The adviewer has served
over 29 million ads.

I will try disabling the banner ads for now and set them so they are
querying from a completely different server, maybe using mongodb and node.js


On Thu, May 10, 2012 at 11:28 AM, pbreit  wrote:

> Is your traffic from serving ads or users coming to your web site? Have
> you exhausted caching opportunities?




-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

[web2py] Re: Javascript file in static folder not refreshed

[Niphlod]

Expiring headers wouldn't be smart what if you changed the html 
trusting that the new javascript will be fetched instead of the cached one ?
Or, you choose to set expire headers 2 days in the future, and you need to 
make a change to the code, but need to wait 2 days to wait for 
expiration.
the only solution is versioning static files.

e.g. /static/js/1.2.3/mysnippet.js vs /static/js/1.2.4/mysnippet.js
or
/static/js/mysnippet.js?ver=1.2.3 vs /static/js/mysnippet.js?ver=1.2.4



Il giorno martedì 8 maggio 2012 21:25:26 UTC+2, Derek ha scritto:
>
> That would fix it for the one system - but if you have site that is used 
> by many, are you going to have everyone empty their cache? Isn't there a 
> way to set the cache headers?
>
> On Tuesday, May 8, 2012 10:34:30 AM UTC-7, Anthony wrote:
>>
>> It's probably being cached by the browser, so you may need to clear the 
>> browser cache (simply refreshing the page won't do it).
>>
>> Anthony
>>
>> On Tuesday, May 8, 2012 1:11:51 PM UTC-4, monotasker wrote:
>>>
>>> I'm working on a js file that resides in appname/static/js/ but for some 
>>> reason it seems to be cached. When I refresh the browser (even if I restart 
>>> the local web2py server) the page continues to load an old version. I 
>>> develop largely with custom modules, and those files are refreshing just 
>>> fine. I have this in my db.py model file:
>>>
>>> from gluon.custom_import import track_changes
>>> track_changes(True)
>>>
>>> But for some reason this javascript file doesn't want to refresh. Any 
>>> suggestions?
>>>
>>> Ian
>>>
>>

Re: [web2py] Need a little help in code review (a function eating up all memory)

[Bruce Wade]

Sorry, you really need to read more about how python works. If you learn
how for loops work and memory you will understand the problem.

One solution do the query before the for loop then loop through the
objects. This may help at bit. Research xrange vs range


On Thu, May 10, 2012 at 11:30 AM, szimszon  wrote:

> Sorry I don't understand. What do you mean "achieve with join"?
>
> There is an empty for loop with db.executesql() without join. And it is
> eating up the memory. :(
>
> 2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a következőt
> írta:
>
>> You can't manage what you want to achieve with join?
>>
>> Richard
>>
>> On Thu, May 10, 2012 at 10:48 AM, szimszon  wrote:
>>
>>> Sorry for my dumbness but if something is wrong with my code please
>>> point me the right line. I'm not so good in English if it comes to "object
>>> instance count" and so. Yeah I know I should go and do some milkmaid job :)
>>> but I'm curious.
>>>
>>> I'm just define some variable:
>>>
>>> lista = list()
>>> last_row = None
>>> next_page_number = 0
>>>
>>> Go in a for loop that just assign the db query result one-by-one to row
>>> variable.
>>>
>>> After that I assign the TABLE() helper to a list variable. That is it.
>>>
>>>
>>>
>>> 2012. május 10., csütörtök 16:34:49 UTC+2 időpontban Bruce Wade a
>>> következőt írta:
>>>
 Using direct sql query or DAL is going to cause the exact same problem
 in this situation.

 On Thu, May 10, 2012 at 7:28 AM, szimszon  wrote:

> It's postgres:// in a
> Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and
>
> Python 2.7.3 (default, Apr 20 2012, 22:44:07)
> [GCC 4.6.3] on linux2
>
> python-psycopg2 2.4.5-1
>
>
> 2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a
> következőt írta:
>
>> Just for curiosity, what happens if you do it in pure sql?
>>
>>
>> for row in db.executesql("**select * from file_properties where
>> id > 0"):
>>
>> # do something
>>
>>
>> Does it have a lower memory usage?
>>
>>
>> On Thu, May 10, 2012 at 4:14 AM, Bruce Wade < 
>> >wrote:
>>
>>> for row in db( db.file_**properties.id > 0 ).select(
>>
>>
>>
>>
>> --
>>
>> Bruno Rocha
>> [http://rochacbruno.com.br]
>>
>>


 --
 --
 Regards,
 Bruce Wade
 http://ca.linkedin.com/in/**bruc**elwade
 http://www.wadecybertech.com
 http://www.fittraineronline.**co**m - 
 Fitness Personal Trainers Online
 http://www.warplydesigned.com


>>


-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

Re: [web2py] Need a little help in code review (a function eating up all memory)

[szimszon]

Is there a doc already known by you explaining this? Can you post a link?

2012. május 10., csütörtök 20:42:22 UTC+2 időpontban Bruce Wade a 
következőt írta:
>
> Sorry, you really need to read more about how python works. If you learn 
> how for loops work and memory you will understand the problem. 
>
> One solution do the query before the for loop then loop through the 
> objects. This may help at bit. Research xrange vs range
>
>
> On Thu, May 10, 2012 at 11:30 AM, szimszon  wrote:
>
>> Sorry I don't understand. What do you mean "achieve with join"?
>>
>> There is an empty for loop with db.executesql() without join. And it is 
>> eating up the memory. :(
>>
>> 2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a következőt 
>> írta:
>>
>>> You can't manage what you want to achieve with join?
>>>
>>> Richard
>>>
>>> On Thu, May 10, 2012 at 10:48 AM, szimszon  wrote:
>>>
 Sorry for my dumbness but if something is wrong with my code please 
 point me the right line. I'm not so good in English if it comes to "object 
 instance count" and so. Yeah I know I should go and do some milkmaid job 
 :) 
 but I'm curious.

 I'm just define some variable:

 lista = list()
 last_row = None
 next_page_number = 0

 Go in a for loop that just assign the db query result one-by-one to row 
 variable.

 After that I assign the TABLE() helper to a list variable. That is it.



 2012. május 10., csütörtök 16:34:49 UTC+2 időpontban Bruce Wade a 
 következőt írta:

> Using direct sql query or DAL is going to cause the exact same problem 
> in this situation.
>
> On Thu, May 10, 2012 at 7:28 AM, szimszon  wrote:
>
>> It's postgres:// in a 
>> Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and
>>
>> Python 2.7.3 (default, Apr 20 2012, 22:44:07) 
>> [GCC 4.6.3] on linux2
>>
>> python-psycopg2 2.4.5-1
>>
>>
>> 2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a 
>> következőt írta:
>>
>>> Just for curiosity, what happens if you do it in pure sql?
>>>
>>>
>>> for row in db.executesql("**select * from file_properties where 
>>> id > 0"):
>>>
>>> # do something
>>>
>>>
>>> Does it have a lower memory usage?
>>>
>>>
>>> On Thu, May 10, 2012 at 4:14 AM, Bruce Wade < 
>>> > wrote:
>>>
 for row in db( db.file_**properties.id > 0 ).select(
>>>
>>>
>>>
>>>
>>> -- 
>>>
>>> Bruno Rocha
>>> [http://rochacbruno.com.br]
>>>
>>>
>
>
> -- 
> -- 
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/**bruc**elwade
> http://www.wadecybertech.com
> http://www.fittraineronline.**co**m - 
> Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>
>>>
>
>
> -- 
> -- 
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
> 

Re: [web2py] Combination of fields must be unique

[Alec Taylor]

On Thursday, May 10, 2012 8:36:18 PM UTC+10, rochacbruno wrote:
>
> db.define_table('nm', Field('a', notnull=True),  Field('b', notnull=True), 
> Field('unikey', unique=True))
>
> db.nm.unikey.compute = lambda row: row.a + row.b
>
Thanks, but that didn't work. I am able to push in multiple duplicates into 
the db.

Re: [web2py] Need a little help in code review (a function eating up all memory)

[szimszon]

In book it is a recommended way to iterate over sql results:

http://web2py.com/books/default/chapter/29/6

You can do all the steps in one statement:

1.
2.
3.

>>> for row in db(db.person.name=='Alex').select():
print row.name
Alex




2012. május 10., csütörtök 20:42:22 UTC+2 időpontban Bruce Wade a 
következőt írta:
>
> Sorry, you really need to read more about how python works. If you learn 
> how for loops work and memory you will understand the problem. 
>
> One solution do the query before the for loop then loop through the 
> objects. This may help at bit. Research xrange vs range
>
>
> On Thu, May 10, 2012 at 11:30 AM, szimszon  wrote:
>
>> Sorry I don't understand. What do you mean "achieve with join"?
>>
>> There is an empty for loop with db.executesql() without join. And it is 
>> eating up the memory. :(
>>
>> 2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a következőt 
>> írta:
>>
>>> You can't manage what you want to achieve with join?
>>>
>>> Richard
>>>
>>> On Thu, May 10, 2012 at 10:48 AM, szimszon  wrote:
>>>
 Sorry for my dumbness but if something is wrong with my code please 
 point me the right line. I'm not so good in English if it comes to "object 
 instance count" and so. Yeah I know I should go and do some milkmaid job 
 :) 
 but I'm curious.

 I'm just define some variable:

 lista = list()
 last_row = None
 next_page_number = 0

 Go in a for loop that just assign the db query result one-by-one to row 
 variable.

 After that I assign the TABLE() helper to a list variable. That is it.



 2012. május 10., csütörtök 16:34:49 UTC+2 időpontban Bruce Wade a 
 következőt írta:

> Using direct sql query or DAL is going to cause the exact same problem 
> in this situation.
>
> On Thu, May 10, 2012 at 7:28 AM, szimszon  wrote:
>
>> It's postgres:// in a 
>> Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and
>>
>> Python 2.7.3 (default, Apr 20 2012, 22:44:07) 
>> [GCC 4.6.3] on linux2
>>
>> python-psycopg2 2.4.5-1
>>
>>
>> 2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a 
>> következőt írta:
>>
>>> Just for curiosity, what happens if you do it in pure sql?
>>>
>>>
>>> for row in db.executesql("**select * from file_properties where 
>>> id > 0"):
>>>
>>> # do something
>>>
>>>
>>> Does it have a lower memory usage?
>>>
>>>
>>> On Thu, May 10, 2012 at 4:14 AM, Bruce Wade < 
>>> > wrote:
>>>
 for row in db( db.file_**properties.id > 0 ).select(
>>>
>>>
>>>
>>>
>>> -- 
>>>
>>> Bruno Rocha
>>> [http://rochacbruno.com.br]
>>>
>>>
>
>
> -- 
> -- 
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/**bruc**elwade
> http://www.wadecybertech.com
> http://www.fittraineronline.**co**m - 
> Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>
>>>
>
>
> -- 
> -- 
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
> 

Re: [web2py] Need a little help in code review (a function eating up all memory)

[Richard Vézina]

I didn't read enough your logic, but since it was implicated a other table
I thougth you just want to do a kind of computed field.

Also, what you seems to do is a kind of paging fonction, why you can't
 achieve this with count()?

Richard

On Thu, May 10, 2012 at 2:30 PM, szimszon  wrote:

> Sorry I don't understand. What do you mean "achieve with join"?
>
> There is an empty for loop with db.executesql() without join. And it is
> eating up the memory. :(
>
> 2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a következőt
> írta:
>
>> You can't manage what you want to achieve with join?
>>
>> Richard
>>
>> On Thu, May 10, 2012 at 10:48 AM, szimszon  wrote:
>>
>>> Sorry for my dumbness but if something is wrong with my code please
>>> point me the right line. I'm not so good in English if it comes to "object
>>> instance count" and so. Yeah I know I should go and do some milkmaid job :)
>>> but I'm curious.
>>>
>>> I'm just define some variable:
>>>
>>> lista = list()
>>> last_row = None
>>> next_page_number = 0
>>>
>>> Go in a for loop that just assign the db query result one-by-one to row
>>> variable.
>>>
>>> After that I assign the TABLE() helper to a list variable. That is it.
>>>
>>>
>>>
>>> 2012. május 10., csütörtök 16:34:49 UTC+2 időpontban Bruce Wade a
>>> következőt írta:
>>>
 Using direct sql query or DAL is going to cause the exact same problem
 in this situation.

 On Thu, May 10, 2012 at 7:28 AM, szimszon  wrote:

> It's postgres:// in a
> Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and
>
> Python 2.7.3 (default, Apr 20 2012, 22:44:07)
> [GCC 4.6.3] on linux2
>
> python-psycopg2 2.4.5-1
>
>
> 2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a
> következőt írta:
>
>> Just for curiosity, what happens if you do it in pure sql?
>>
>>
>> for row in db.executesql("**select * from file_properties where
>> id > 0"):
>>
>> # do something
>>
>>
>> Does it have a lower memory usage?
>>
>>
>> On Thu, May 10, 2012 at 4:14 AM, Bruce Wade < 
>> >wrote:
>>
>>> for row in db( db.file_**properties.id > 0 ).select(
>>
>>
>>
>>
>> --
>>
>> Bruno Rocha
>> [http://rochacbruno.com.br]
>>
>>


 --
 --
 Regards,
 Bruce Wade
 http://ca.linkedin.com/in/**bruc**elwade
 http://www.wadecybertech.com
 http://www.fittraineronline.**co**m - 
 Fitness Personal Trainers Online
 http://www.warplydesigned.com


>>

Re: [web2py] Need a little help in code review (a function eating up all memory)

[Richard Vézina]

Yes but in this case it is not for the entire records...

Why would you return a full list of all the records?

I don't understand what is the purpose of listar that you return in the
view under a html table, why do you need to return all the 10+ entries?

Richard

On Thu, May 10, 2012 at 2:56 PM, szimszon  wrote:

> In book it is a recommended way to iterate over sql results:
>
> http://web2py.com/books/default/chapter/29/6
>
> You can do all the steps in one statement:
>
> 1.
> 2.
> 3.
>
> >>> for row in db(db.person.name=='Alex').select():
> print row.name
> Alex
>
>
>
>
> 2012. május 10., csütörtök 20:42:22 UTC+2 időpontban Bruce Wade a
> következőt írta:
>
>> Sorry, you really need to read more about how python works. If you learn
>> how for loops work and memory you will understand the problem.
>>
>> One solution do the query before the for loop then loop through the
>> objects. This may help at bit. Research xrange vs range
>>
>>
>> On Thu, May 10, 2012 at 11:30 AM, szimszon  wrote:
>>
>>> Sorry I don't understand. What do you mean "achieve with join"?
>>>
>>> There is an empty for loop with db.executesql() without join. And it is
>>> eating up the memory. :(
>>>
>>> 2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a
>>> következőt írta:
>>>
 You can't manage what you want to achieve with join?

 Richard

 On Thu, May 10, 2012 at 10:48 AM, szimszon  wrote:

> Sorry for my dumbness but if something is wrong with my code please
> point me the right line. I'm not so good in English if it comes to "object
> instance count" and so. Yeah I know I should go and do some milkmaid job 
> :)
> but I'm curious.
>
> I'm just define some variable:
>
> lista = list()
> last_row = None
> next_page_number = 0
>
> Go in a for loop that just assign the db query result one-by-one to
> row variable.
>
> After that I assign the TABLE() helper to a list variable. That is it.
>
>
>
> 2012. május 10., csütörtök 16:34:49 UTC+2 időpontban Bruce Wade a
> következőt írta:
>
>> Using direct sql query or DAL is going to cause the exact same
>> problem in this situation.
>>
>> On Thu, May 10, 2012 at 7:28 AM, szimszon  wrote:
>>
>>> It's postgres:// in a
>>> Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and
>>>
>>> Python 2.7.3 (default, Apr 20 2012, 22:44:07)
>>> [GCC 4.6.3] on linux2
>>>
>>> python-psycopg2 2.4.5-1
>>>
>>>
>>> 2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a
>>> következőt írta:
>>>
 Just for curiosity, what happens if you do it in pure sql?


 for row in db.executesql("**sele**ct * from file_properties
 where id > 0"):

 # do something


 Does it have a lower memory usage?


 On Thu, May 10, 2012 at 4:14 AM, Bruce Wade <
 > wrote:

> for row in db( db.file_**propert**ies.id > 0 ).select(




 --

 Bruno Rocha
 [http://rochacbruno.com.br]


>>
>>
>> --
>> --
>> Regards,
>> Bruce Wade
>> http://ca.linkedin.com/in/**brucelwade
>> http://www.wadecybertech.com
>> http://www.fittraineronline.**com- 
>> Fitness Personal Trainers Online
>> http://www.warplydesigned.com
>>
>>

>>
>>
>> --
>> --
>> Regards,
>> Bruce Wade
>> http://ca.linkedin.com/in/**brucelwade
>> http://www.wadecybertech.com
>> http://www.fittraineronline.**com  -
>> Fitness Personal Trainers Online
>> http://www.warplydesigned.com
>>
>>

[web2py] Centralised CRUD form for all referenced tables

[Alec Taylor]

I have 4 tables which I would like to create a form for.

As a test-case I have greatly simplified them and only included two:

db.define_table('product',
Field('name','string'),
Field('category','string')
)

db.define_table('product_list',
Field('product_id', 'list:reference db.product'),
Field('category', 'string')
)

db.product_list.product_id.requires = IS_IN_DB(db, db.product.id, '%(name)s'
, multiple=True)

The form should have:

   - an editable list allowing for modification of product_list and product
   - a read only html table showing the nesting
   - a create new product list form allowing you to not only create new 
   products for inclusion but also to choose other products for inclusion

How would I create this using the in-built crud library?

Thanks for all suggestions,

Alec Taylor

[web2py] PyCon Argentina 2012: Call for Proposals: talks, tutorials, scientific papers and student works contest

[Mariano Reingart]

PyCon Argentina 2012 - Fourth Spanish-speaking National Python Conference
November 12th - 17th, 2012
UNQ - National University of Quilmes - Buenos Aires

http://ar.pycon.org/?lang=en

PyAr, the Argentina Python User Group, is proud to present the 4th
annual PyCon Argentina, in Buenos Aires this year again.

The event is expected to join around 500 developers, technologists,
programmers and hackers; bloggers, authors and web developers; CTOs,
managers and entrepreneurs; scientists, engineers and domain experts,
and free software community in general.

We're especially interested in 40' presentations and 120' Tutorials
that will teach conference-goers something new and useful.
Other presentation oportunities includes:

 * Posters: http://ar.pycon.org/2012/conference/posters?lang=en
 * Scientific Papers: http://ar.pycon.org/2012/conference/science?lang=en
 * Student Works: http://ar.pycon.org/2012/conference/contest?lang=en
 * Sprint Projects: http://ar.pycon.org/2012/conference/sprints?lang=en


Can you show attendees how to use a module? Explore a Python language
feature? Package an application?
Do you study with Python? Do you research with Python?
Submit your activity proposal here:

http://ar.pycon.org/2012/conference/proposals/?lang=en

Please follow the instructions carefully.

The proposals should be submitted preferably in Spanish (although
English and Portuguese are allowed languages too).

Submission deadline is June 30th, 2012 23:59 ART (GMT-03)

PyCon Argentina is organized and run by volunteers from the Python community.
Anyone can help, and all help is very much appreciated.
If you blog, please let your readers know about PyCon.
A blog badge is a nice way to enhance such a post:
http://ar.pycon.org/2012/conference/publicize/

PyCon Argentina is a free (as in beer) community conference.
We keep the costs low to make PyCon Argentina accessible to the
greatest range of community members possible.
This means that PyCon Argentina cannot offer payment to conference speakers.
Conference registration, accommodations, and transportation are the
responsibility of the speakers.

However, financial aid will be available, and speakers who pay their
own way are encouraged to apply.
Giving a talk is a great rationale when requesting aid.
If you require financial aid in order to attend PyCon Argentina,
please note it in your proposal and be sure to apply.

Pre-Confirmed International invited speakers includes so far:
 * Massimo Di Pierro - De Paul University - web2py / FermiQCD
 * Andrea Gavana - Maersk Oil - wx.lib.agw & 2D/3D visualization
 * Brett Cannon - Google - python core developer: importlib py3k pypy

Sponsorship opportunities are open, for further information see:
http://ar.pycon.org/2012/sponsors/prospectus?lang=en

Free and Open Source projects will have booths or special events
inside the conference, please contact us if your community is
interested.


Mariano Reingart
PyCon Argentina 2012 Chair
http://ar.pycon.org/2012

Re: [web2py] Need a little help in code review (a function eating up all memory)

[szimszon]

I had to store files and a lot of properties for it. It was in csv. But 
after I processed it we figured out that not all value was correct in csv 
but it was a bit redundant. So I can correct it by go through all the 
records row by row. So that was a one time trip.

I just realized after the process I had no memory left. So now I'm 
investigating what happened... 

2012. május 10., csütörtök 21:00:05 UTC+2 időpontban Richard a következőt 
írta:
>
> Yes but in this case it is not for the entire reecords...
>
> Why would you return a full list of all the records?
>
> I don't understand what is the purpose of listar that you return in the 
> view under a html table, why do you need to return all the 10+ entries?
>
> Richard
>
> On Thu, May 10, 2012 at 2:56 PM, szimszon  wrote:
>
>> In book it is a recommended way to iterate over sql results:
>>
>> http://web2py.com/books/default/chapter/29/6
>>
>> You can do all the steps in one statement:
>>
>> 1.
>> 2.
>> 3.
>>
>> >>> for row in db(db.person.name=='Alex').select():
>>
>> print row.name
>> Alex
>>
>>
>>
>>
>> 2012. május 10., csütörtök 20:42:22 UTC+2 időpontban Bruce Wade a 
>> következőt írta:
>>
>>> Sorry, you really need to read more about how python works. If you learn 
>>> how for loops work and memory you will understand the problem. 
>>>
>>> One solution do the query before the for loop then loop through the 
>>> objects. This may help at bit. Research xrange vs range
>>>
>>>
>>> On Thu, May 10, 2012 at 11:30 AM, szimszon  wrote:
>>>
 Sorry I don't understand. What do you mean "achieve with join"?

 There is an empty for loop with db.executesql() without join. And it is 
 eating up the memory. :(

 2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a 
 következőt írta:

> You can't manage what you want to achieve with join?
>
> Richard
>
> On Thu, May 10, 2012 at 10:48 AM, szimszon  wrote:
>
>> Sorry for my dumbness but if something is wrong with my code please 
>> point me the right line. I'm not so good in English if it comes to 
>> "object 
>> instance count" and so. Yeah I know I should go and do some milkmaid job 
>> :) 
>> but I'm curious.
>>
>> I'm just define some variable:
>>
>> lista = list()
>> last_row = None
>> next_page_number = 0
>>
>> Go in a for loop that just assign the db query result one-by-one to 
>> row variable.
>>
>> After that I assign the TABLE() helper to a list variable. That is it.
>>
>>
>>
>> 2012. május 10., csütörtök 16:34:49 UTC+2 időpontban Bruce Wade a 
>> következőt írta:
>>
>>> Using direct sql query or DAL is going to cause the exact same 
>>> problem in this situation.
>>>
>>> On Thu, May 10, 2012 at 7:28 AM, szimszon wrote:
>>>
 It's postgres:// in a 
 Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and

 Python 2.7.3 (default, Apr 20 2012, 22:44:07) 
 [GCC 4.6.3] on linux2

 python-psycopg2 2.4.5-1


 2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a 
 következőt írta:

> Just for curiosity, what happens if you do it in pure sql?
>
>
> for row in db.executesql("**sele**ct * from file_properties 
> where id > 0"):
>
> # do something
>
>
> Does it have a lower memory usage?
>
>
> On Thu, May 10, 2012 at 4:14 AM, Bruce Wade <
> > wrote:
>
>> for row in db( db.file_**propert**ies.id > 0 ).select(
>
>
>
>
> -- 
>
> Bruno Rocha
> [http://rochacbruno.com.br]
>
>
>>>
>>>
>>> -- 
>>> -- 
>>> Regards,
>>> Bruce Wade
>>> http://ca.linkedin.com/in/**brucelwade
>>> http://www.wadecybertech.com
>>> http://www.fittraineronline.**com- 
>>> Fitness Personal Trainers Online
>>> http://www.warplydesigned.com
>>>
>>>
>
>>>
>>>
>>> -- 
>>> -- 
>>> Regards,
>>> Bruce Wade
>>> http://ca.linkedin.com/in/**brucelwade
>>> http://www.wadecybertech.com
>>> http://www.fittraineronline.**com  - 
>>> Fitness Personal Trainers Online
>>> http://www.warplydesigned.com
>>>
>>>  
>

[web2py] Re: External database query for authentication (RBAC)

[Massimo Di Pierro]

In the scaffolding app there is a file db.py which defines for you 
everything you need for RBAC. If you do not want to use this built-in 
functionality simply delete the file and use your own API. There is nothing 
that really depends on it.

On Thursday, 10 May 2012 08:17:35 UTC-5, BoleroDan wrote:
>
> We're looking to use Web2py as our framework for developing a couple 
> applications. However we already have a massive database for our research 
> project, and almost all of the information is integrated into this database 
> (including usernames, passwords, group names etc) so it would be nice to 
> have web2py just access this database for username/password/group 
> membership authentication.
>
> I'm hoping this can be integrated into Web2Pys already existing RBAC 
> management as that would be nice. I also noticed that a lot of the other 
> alternative login methods still require web2pys local auth_user table and 
> stores the remote credentials locally in this table. Is this needed by 
> design or can this be skipped? I'm just thinking its not needed for us to 
> have duplicate login information across two databases. I just need Web2py 
> to query our database, get the credentials authenticated and what group 
> they are in (so i can control access to functions on web2py based on their 
> group).
>
> Thanks for any heads up on the matter.
>

[web2py] Re: jqgrid assistance

[Massimo Di Pierro]

If you use chrome check the developer tools. There should be some ajax 
request to get data. Does it fail? Why?
To check you can also delete your tickets in errors/* and see if any new 
one shows up.

On Thursday, 10 May 2012 08:31:15 UTC-5, Larry Wapnitsky wrote:
>
> I'm trying out jqgrid (yes, I finally got plugin_wiki working ;)), but 
> can't seem to get it to display data.  Here is my code:
>
> *{{=plugin_wiki.widget('jqgrid', db.ips)}}*
>
> I see a table with MOST of my fields showing (not all, strangely), but no 
> data.  I know for a fact that the database is populated.
>
> Ideas?
>
> Thanks
>

[web2py] Re: jqgrid assistance

[Larry Wapnitsky]

This is what's showing up in the source code:

http://pastie.textmate.org/3891282

On Thursday, May 10, 2012 3:44:49 PM UTC-4, Massimo Di Pierro wrote:
>
> If you use chrome check the developer tools. There should be some ajax 
> request to get data. Does it fail? Why?
> To check you can also delete your tickets in errors/* and see if any new 
> one shows up.
>
> On Thursday, 10 May 2012 08:31:15 UTC-5, Larry Wapnitsky wrote:
>>
>> I'm trying out jqgrid (yes, I finally got plugin_wiki working ;)), but 
>> can't seem to get it to display data.  Here is my code:
>>
>> *{{=plugin_wiki.widget('jqgrid', db.ips)}}*
>>
>> I see a table with MOST of my fields showing (not all, strangely), but no 
>> data.  I know for a fact that the database is populated.
>>
>> Ideas?
>>
>> Thanks
>>
>
On Thursday, May 10, 2012 3:44:49 PM UTC-4, Massimo Di Pierro wrote:
>
> If you use chrome check the developer tools. There should be some ajax 
> request to get data. Does it fail? Why?
> To check you can also delete your tickets in errors/* and see if any new 
> one shows up.
>
> On Thursday, 10 May 2012 08:31:15 UTC-5, Larry Wapnitsky wrote:
>>
>> I'm trying out jqgrid (yes, I finally got plugin_wiki working ;)), but 
>> can't seem to get it to display data.  Here is my code:
>>
>> *{{=plugin_wiki.widget('jqgrid', db.ips)}}*
>>
>> I see a table with MOST of my fields showing (not all, strangely), but no 
>> data.  I know for a fact that the database is populated.
>>
>> Ideas?
>>
>> Thanks
>>
>

Re: [web2py] Need a little help in code review (a function eating up all memory)

[Richard Vézina]

Ok, you don't need it to works all the time.

Did you get it to update your records?

If not, and if as you said it is a one trip, you can just treat the whole
records batch by batch...

Look here :
http://web2py.com/books/default/chapter/29/14#Populating-database-with-dummy-data

for i in range(10):
populate(db.mytable,100)
db.commit()


In this example only 100 records are populated at a time between
db.commit()... So maybe you just have to wrap for loop that will
db.commit() a couples of time during your processing.

Richard


On Thu, May 10, 2012 at 3:32 PM, szimszon  wrote:

> I had to store files and a lot of properties for it. It was in csv. But
> after I processed it we figured out that not all value was correct in csv
> but it was a bit redundant. So I can correct it by go through all the
> records row by row. So that was a one time trip.
>
> I just realized after the process I had no memory left. So now I'm
> investigating what happened...
>
> 2012. május 10., csütörtök 21:00:05 UTC+2 időpontban Richard a következőt
> írta:
>>
>> Yes but in this case it is not for the entire reecords...
>>
>> Why would you return a full list of all the records?
>>
>> I don't understand what is the purpose of listar that you return in the
>> view under a html table, why do you need to return all the 10+ entries?
>>
>> Richard
>>
>> On Thu, May 10, 2012 at 2:56 PM, szimszon  wrote:
>>
>>> In book it is a recommended way to iterate over sql results:
>>>
>>> http://web2py.com/books/**default/chapter/29/6
>>>
>>> You can do all the steps in one statement:
>>>
>>> 1.
>>> 2.
>>> 3.
>>>
>>> >>> for row in db(db.person.name=='Alex').sel**ect():
>>>
>>> print row.name
>>> Alex
>>>
>>>
>>>
>>>
>>> 2012. május 10., csütörtök 20:42:22 UTC+2 időpontban Bruce Wade a
>>> következőt írta:
>>>
 Sorry, you really need to read more about how python works. If you
 learn how for loops work and memory you will understand the problem.

 One solution do the query before the for loop then loop through the
 objects. This may help at bit. Research xrange vs range


 On Thu, May 10, 2012 at 11:30 AM, szimszon  wrote:

> Sorry I don't understand. What do you mean "achieve with join"?
>
> There is an empty for loop with db.executesql() without join. And it
> is eating up the memory. :(
>
> 2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a
> következőt írta:
>
>> You can't manage what you want to achieve with join?
>>
>> Richard
>>
>> On Thu, May 10, 2012 at 10:48 AM, szimszon wrote:
>>
>>> Sorry for my dumbness but if something is wrong with my code please
>>> point me the right line. I'm not so good in English if it comes to 
>>> "object
>>> instance count" and so. Yeah I know I should go and do some milkmaid 
>>> job :)
>>> but I'm curious.
>>>
>>> I'm just define some variable:
>>>
>>> lista = list()
>>> last_row = None
>>> next_page_number = 0
>>>
>>> Go in a for loop that just assign the db query result one-by-one to
>>> row variable.
>>>
>>> After that I assign the TABLE() helper to a list variable. That is
>>> it.
>>>
>>>
>>>
>>> 2012. május 10., csütörtök 16:34:49 UTC+2 időpontban Bruce Wade a
>>> következőt írta:
>>>
 Using direct sql query or DAL is going to cause the exact same
 problem in this situation.

 On Thu, May 10, 2012 at 7:28 AM, szimszon wrote:

> It's postgres:// in a
> Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and
>
> Python 2.7.3 (default, Apr 20 2012, 22:44:07)
> [GCC 4.6.3] on linux2
>
> python-psycopg2 2.4.5-1
>
>
> 2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a
> következőt írta:
>
>> Just for curiosity, what happens if you do it in pure sql?
>>
>>
>> for row in db.executesql("**select * from
>> file_properties where id > 0"):
>>
>> # do something
>>
>>
>> Does it have a lower memory usage?
>>
>>
>> On Thu, May 10, 2012 at 4:14 AM, Bruce Wade <
>> > wrote:
>>
>>> for row in db( db.file_**properties.id > 0 ).select(
>>
>>
>>
>>
>> --
>>
>> Bruno Rocha
>> [http://rochacbruno.com.br]
>>
>>


 --
 --
 Regards,
 Bruce Wade
 http://ca.linkedin.com/in/**bruc**elwade
 http://www.wadecybertech.com
 http://www.fittraineronline.**co**m-
  Fitness Personal Trainers Online
 http://www.warplydesigned.com

>>>

[web2py] Re: Javascript file in static folder not refreshed

[Derek]

I think the second option is the better one. Even better would be to 
support the If-Modified-Since or use ETAG.

On Thursday, May 10, 2012 11:36:01 AM UTC-7, Niphlod wrote:
>
> Expiring headers wouldn't be smart what if you changed the html 
> trusting that the new javascript will be fetched instead of the cached one ?
> Or, you choose to set expire headers 2 days in the future, and you need to 
> make a change to the code, but need to wait 2 days to wait for 
> expiration.
> the only solution is versioning static files.
>
> e.g. /static/js/1.2.3/mysnippet.js vs /static/js/1.2.4/mysnippet.js
> or
> /static/js/mysnippet.js?ver=1.2.3 vs /static/js/mysnippet.js?ver=1.2.4
>
>
>
> Il giorno martedì 8 maggio 2012 21:25:26 UTC+2, Derek ha scritto:
>>
>> That would fix it for the one system - but if you have site that is used 
>> by many, are you going to have everyone empty their cache? Isn't there a 
>> way to set the cache headers?
>>
>> On Tuesday, May 8, 2012 10:34:30 AM UTC-7, Anthony wrote:
>>>
>>> It's probably being cached by the browser, so you may need to clear the 
>>> browser cache (simply refreshing the page won't do it).
>>>
>>> Anthony
>>>
>>> On Tuesday, May 8, 2012 1:11:51 PM UTC-4, monotasker wrote:

 I'm working on a js file that resides in appname/static/js/ but for 
 some reason it seems to be cached. When I refresh the browser (even if I 
 restart the local web2py server) the page continues to load an old 
 version. 
 I develop largely with custom modules, and those files are refreshing just 
 fine. I have this in my db.py model file:

 from gluon.custom_import import track_changes
 track_changes(True)

 But for some reason this javascript file doesn't want to refresh. Any 
 suggestions?

 Ian

>>>

[web2py] Moving away from define_table

[Christopher Baron]

Hi,

I've been using web2py for a few years now.  What I find most annoying is 
the DAL.  I understand that its easy for people who don't have a 
programming background, but anytime I want to make changes to columns - I 
run into issues large and small.  I'd simply prefer to manage my database 
without having web2py altering things.  Please advise.

[web2py] Re: Moving away from define_table

[simon]

Can't you just set migrate=false?

On Thursday, 10 May 2012 20:55:14 UTC+1, Christopher Baron wrote:
>
> Hi,
>
> I've been using web2py for a few years now.  What I find most annoying is 
> the DAL.  I understand that its easy for people who don't have a 
> programming background, but anytime I want to make changes to columns - I 
> run into issues large and small.  I'd simply prefer to manage my database 
> without having web2py altering things.  Please advise.
>

[web2py] Re: jqgrid assistance

[Larry Wapnitsky]

Also just found this via the dev tools::

Request URL:
http://127.0.0.1:8000/rbl_web2py/plugin_wiki/jqgrid?columns=ipaddress%2Cattacknotes%2Cb_or_w%2CupdCount&fieldname=&fieldvalue=None&tablename=ips&_search=false&nd=1336679760463&rows=10&page=1&sidx=&sord=asc

Re: [web2py] Re: Moving away from define_table

[Richard Vézina]

db=DAL('postgres://USER:PWD@127.0.0.1:5432/database', \
*migrate_enabled=False*)

Or as simon wrote :

db.define_table(...,migrate=False)

Richard

On Thu, May 10, 2012 at 4:01 PM, simon  wrote:

> Can't you just set migrate=false?
>
>
> On Thursday, 10 May 2012 20:55:14 UTC+1, Christopher Baron wrote:
>>
>> Hi,
>>
>> I've been using web2py for a few years now.  What I find most annoying is
>> the DAL.  I understand that its easy for people who don't have a
>> programming background, but anytime I want to make changes to columns - I
>> run into issues large and small.  I'd simply prefer to manage my database
>> without having web2py altering things.  Please advise.
>>
>

[web2py] Re: Moving away from define_table

[Anthony]

You can turn off migrations globally for the entire db connection via DAL(..., 
migrate_enabled=False).

Anthony

p.s., Even people with a programming background find the DAL useful. :-)

On Thursday, May 10, 2012 4:01:18 PM UTC-4, simon wrote:
>
> Can't you just set migrate=false?
>
> On Thursday, 10 May 2012 20:55:14 UTC+1, Christopher Baron wrote:
>>
>> Hi,
>>
>> I've been using web2py for a few years now.  What I find most annoying is 
>> the DAL.  I understand that its easy for people who don't have a 
>> programming background, but anytime I want to make changes to columns - I 
>> run into issues large and small.  I'd simply prefer to manage my database 
>> without having web2py altering things.  Please advise.
>>
>

[web2py] Re: jqgrid assistance

[Larry Wapnitsky]

Even this fails on a wiki page:

``
name: jqgrid
table: db.ips
col_width: 80
width: 700
height: 300
``:widget


Traceback (most recent call last):
  File "/home/lwapnitsky/web2py/applications/rbl_web2py/models/plugin_wiki.py", 
line 635, in render_widget
html = getattr(PluginWikiWidgets,name)(**args)
  File "/home/lwapnitsky/web2py/applications/rbl_web2py/models/plugin_wiki.py", 
line 245, in jqgrid
fields = [x.strip() for x in db[table].fields if 
db[table][x.strip()].readable]
  File "/home/lwapnitsky/web2py/gluon/dal.py", line 6773, in __getitem__
return dict.__getitem__(self, str(key))
KeyError: 'db.ips'

[web2py] Re: too many redirect message when using login

[Derek]

from admin, click 'clean'.

On Thursday, May 10, 2012 9:10:58 AM UTC-7, man24 wrote:
>
> Yes, I will open a ticket.  Is there a simpler way of deleting sessions 
> that does not involve command line action and is automated?  I am on mac 
> and using the compiled web2py.
>
> Also, by mistake, I have posted this question twice  -'Problem with 
> auth_login: too many redirects' - so there are now two discussion 
> threads.  I apologize, I am new to this group.  I have just pasted the code 
> for my model, controller and views in that thread, which is almost verbatim 
> from the web2py book.
>
> Thanks
>
> On Thursday, May 10, 2012 8:12:00 AM UTC-5, Massimo Di Pierro wrote:
>>
>> Please open a ticket about this. Meanwhile, can you try delete all 
>> session files? Any progress?
>>
>> On Wednesday, 9 May 2012 23:27:28 UTC-5, man24 wrote:
>>>
>>> I am new to web2py and has been following the online book to learn it. 
>>>  I was following the 'wiki' example and everything worked fine the first 
>>> time, however, today when I click on any action that requires login or even 
>>> the login link in the menu, web browser cannot open the page and gives 
>>> following message:  'too many redirects occurred trying to open the page'. 
>>>  I am using web2py on a mac.  Thanks for your help.
>>
>>

[web2py] Re: MongoDB Adapter error in select

[Massimo Di Pierro]

This helps. I have another attempt to fix this in trunk. Hard to test it 
since I do not have mongodb installed.

On Thursday, 10 May 2012 08:39:22 UTC-5, Francisco Costa wrote:
>
> Still not working. I think Fields order is wrong.
>
> At the end of the select() function I write this:
>
> a = []
> for f in fields:
> a.append(f.type)
> return dict(rows=rows, fields=a, colnames=colnames)
>
> and I got this:
>
> colnames:
> city
> age
> _id
> name
> fields:
> id
> text
> integer
> string
> rows:
> Toronto
> 66L
> 24652490551171733682233802752L
> John
>
>
>

[web2py] Re: jqgrid assistance

[Larry Wapnitsky]

the whole "plugin_wiki" thing is really starting to get my goat.  Now I'm 
getting errors trying to delete my test wiki page: 

 (1452, u'Cannot add or 
update a child row: a foreign key constraint fails 
(`rbl`.`plugin_wiki_page_archive`, CONSTRAINT 
`plugin_wiki_page_archive_ibfk_1` FOREIGN KEY (`current_record`) REFERENCES 
`plugin_wiki_page` (`id`) ON DELETE CASCADE)')
On Thursday, May 10, 2012 3:44:49 PM UTC-4, Massimo Di Pierro wrote:
>
> If you use chrome check the developer tools. There should be some ajax 
> request to get data. Does it fail? Why?
> To check you can also delete your tickets in errors/* and see if any new 
> one shows up.
>
> On Thursday, 10 May 2012 08:31:15 UTC-5, Larry Wapnitsky wrote:
>>
>> I'm trying out jqgrid (yes, I finally got plugin_wiki working ;)), but 
>> can't seem to get it to display data.  Here is my code:
>>
>> *{{=plugin_wiki.widget('jqgrid', db.ips)}}*
>>
>> I see a table with MOST of my fields showing (not all, strangely), but no 
>> data.  I know for a fact that the database is populated.
>>
>> Ideas?
>>
>> Thanks
>>
>

[web2py] Re: Javascript file in static folder not refreshed

[Anthony]

When web2py serves static files, it does set the Last-Modified header, but 
I'm not sure that's enough to prompt an If-Modified-Since request -- we may 
need to add a max-age=0 and/or must-revalidate directive as well.

Anthony

On Thursday, May 10, 2012 3:54:54 PM UTC-4, Derek wrote:
>
> I think the second option is the better one. Even better would be to 
> support the If-Modified-Since or use ETAG.
>
> On Thursday, May 10, 2012 11:36:01 AM UTC-7, Niphlod wrote:
>>
>> Expiring headers wouldn't be smart what if you changed the html 
>> trusting that the new javascript will be fetched instead of the cached one ?
>> Or, you choose to set expire headers 2 days in the future, and you need 
>> to make a change to the code, but need to wait 2 days to wait for 
>> expiration.
>> the only solution is versioning static files.
>>
>> e.g. /static/js/1.2.3/mysnippet.js vs /static/js/1.2.4/mysnippet.js
>> or
>> /static/js/mysnippet.js?ver=1.2.3 vs /static/js/mysnippet.js?ver=1.2.4
>>
>>
>>
>> Il giorno martedì 8 maggio 2012 21:25:26 UTC+2, Derek ha scritto:
>>>
>>> That would fix it for the one system - but if you have site that is used 
>>> by many, are you going to have everyone empty their cache? Isn't there a 
>>> way to set the cache headers?
>>>
>>> On Tuesday, May 8, 2012 10:34:30 AM UTC-7, Anthony wrote:

 It's probably being cached by the browser, so you may need to clear the 
 browser cache (simply refreshing the page won't do it).

 Anthony

 On Tuesday, May 8, 2012 1:11:51 PM UTC-4, monotasker wrote:
>
> I'm working on a js file that resides in appname/static/js/ but for 
> some reason it seems to be cached. When I refresh the browser (even if I 
> restart the local web2py server) the page continues to load an old 
> version. 
> I develop largely with custom modules, and those files are refreshing 
> just 
> fine. I have this in my db.py model file:
>
> from gluon.custom_import import track_changes
> track_changes(True)
>
> But for some reason this javascript file doesn't want to refresh. Any 
> suggestions?
>
> Ian
>

Re: [web2py] Need a little help in code review (a function eating up all memory)

[szimszon]

Okay. It's clear.

I'm only puzzled about why the memory didn't get freed or reused after 
execution is finished. And if I execute the controller function in 1-2min 
interval mem is still not reused.

So I understand it can eat up the memory but why is all memory locked 
forever and didn't get reused - in my understanding (it's not much) in my 
case GC do not free mem at all in python level. So some reference is still 
intact after function is finished.


2012. május 10., csütörtök 21:53:06 UTC+2 időpontban Richard a következőt 
írta:
>
> Ok, you don't need it to works all the time.
>
> Did you get it to update your records?
>
> If not, and if as you said it is a one trip, you can just treat the whole 
> records batch by batch...
>
> Look here : 
> http://web2py.com/books/default/chapter/29/14#Populating-database-with-dummy-data
>
> for i in range(10):
>
> populate(db.mytable,100)
>
> db.commit()
>
>
> In this example only 100 records are populated at a time between 
> db.commit()... So maybe you just have to wrap for loop that will 
> db.commit() a couples of time during your processing.
>
> Richard
>
>
> On Thu, May 10, 2012 at 3:32 PM, szimszon  wrote:
>
>> I had to store files and a lot of properties for it. It was in csv. But 
>> after I processed it we figured out that not all value was correct in csv 
>> but it was a bit redundant. So I can correct it by go through all the 
>> records row by row. So that was a one time trip.
>>
>> I just realized after the process I had no memory left. So now I'm 
>> investigating what happened... 
>>
>> 2012. május 10., csütörtök 21:00:05 UTC+2 időpontban Richard a következőt 
>> írta:
>>>
>>> Yes but in this case it is not for the entire reecords...
>>>
>>> Why would you return a full list of all the records?
>>>
>>> I don't understand what is the purpose of listar that you return in the 
>>> view under a html table, why do you need to return all the 10+ entries?
>>>
>>> Richard
>>>
>>> On Thu, May 10, 2012 at 2:56 PM, szimszon  wrote:
>>>
 In book it is a recommended way to iterate over sql results:

 http://web2py.com/books/**default/chapter/29/6

 You can do all the steps in one statement:

 1.
 2.
 3.

 >>> for row in db(db.person.name=='Alex').sel**ect():


 print row.name

 Alex




 2012. május 10., csütörtök 20:42:22 UTC+2 időpontban Bruce Wade a 
 következőt írta:

> Sorry, you really need to read more about how python works. If you 
> learn how for loops work and memory you will understand the problem. 
>
> One solution do the query before the for loop then loop through the 
> objects. This may help at bit. Research xrange vs range
>
>
> On Thu, May 10, 2012 at 11:30 AM, szimszon  wrote:
>
>> Sorry I don't understand. What do you mean "achieve with join"?
>>
>> There is an empty for loop with db.executesql() without join. And it 
>> is eating up the memory. :(
>>
>> 2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a 
>> következőt írta:
>>
>>> You can't manage what you want to achieve with join?
>>>
>>> Richard
>>>
>>> On Thu, May 10, 2012 at 10:48 AM, szimszon wrote:
>>>
 Sorry for my dumbness but if something is wrong with my code please 
 point me the right line. I'm not so good in English if it comes to 
 "object 
 instance count" and so. Yeah I know I should go and do some milkmaid 
 job :) 
 but I'm curious.

 I'm just define some variable:

 lista = list()
 last_row = None
 next_page_number = 0

 Go in a for loop that just assign the db query result one-by-one to 
 row variable.

 After that I assign the TABLE() helper to a list variable. That is 
 it.



 2012. május 10., csütörtök 16:34:49 UTC+2 időpontban Bruce Wade a 
 következőt írta:

> Using direct sql query or DAL is going to cause the exact same 
> problem in this situation.
>
> On Thu, May 10, 2012 at 7:28 AM, szimszon wrote:
>
>> It's postgres:// in a 
>> Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and
>>
>> Python 2.7.3 (default, Apr 20 2012, 22:44:07) 
>> [GCC 4.6.3] on linux2
>>
>> python-psycopg2 2.4.5-1
>>
>>
>> 2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno 
>> a következőt írta:
>>
>>> Just for curiosity, what happens if you do it in pure sql?
>>>
>>>
>>> for row in db.executesql("**select * from 
>>> file_properties where id > 0"):
>>>
>>> # do something
>>>
>>>
>>> Does it have a low

[web2py] Re: Problem with auth_login: too many redirects

[Anthony]

@auth.requires_login()
def user():

Don't require login to get to the user() function -- if you're not logged 
in, it redirects to the user() function, which will result in an infinite 
redirect loop.

Anthony

On Thursday, May 10, 2012 11:33:34 AM UTC-4, man24 wrote:
>
> Here is the model file - > db2.py 
>
> # coding: utf8
> db.define_table('page',
> Field('title', requires=IS_NOT_IN_DB(db, 'page.title')),
> Field('body', 'text', requires=IS_NOT_EMPTY()),
> Field('created_on', 'datetime', default=request.now, 
> readable=False, writable=False),
> Field('created_by', db.auth_user, default=auth.user_id, 
> readable=False, writable=False),
> format='%(title)s')
> 
> db.define_table('comment',
> Field('page_id', db.page, readable=False, writable=False),
> Field('body', 'text', requires=IS_NOT_EMPTY()),
> Field('created_on', 'datetime', default=request.now, 
> readable=False, writable=False),
> Field('created_by', db.auth_user, 
> default=auth.user_id,readable=False, writable=False))
> 
> db.define_table('document',
> Field('page_id', db.page, readable=False, writable=False),
> Field('name', requires=IS_NOT_IN_DB(db, 'document.name')),
> Field('file', 'upload'),
> Field('created_on', 'datetime', default=request.now, 
> readable=False, writable=False),
> Field('created_by', db.auth_user, default=auth.user_id, 
> readable=False, writable=False),
> format='%(name)s')
>
> Here is the controller: 
>
> def index():
> pages = db().select(db.page.id, db.page.title, orderby=db.page.title)
> return dict(pages=pages)
> 
> @auth.requires_login()
> def create():
> form=crud.create(db.page, next=URL('index'))
> return dict(form=form)
> 
> def show():
> this_page = db.page(request.args(0)) or redirect(URL('index'))
> db.comment.page_id.default = this_page.id
> form = crud.create(db.comment) if auth.user else None
> pagecomments = db(db.comment.page_id==this_page.id).select()
> return dict(page=this_page, comments=pagecomments, form=form)
>
> @auth.requires_login()
> def edit():
> this_page = db.page(request.args(0)) or redirect(URL('index'))
> form = crud.update(db.page, this_page, next=URL('show', 
> args=request.args))
> return dict(form=form)
> 
> @auth.requires_login()
> def documents():
> page = db.page(request.args(0)) or redirect(URL('index'))
> db.document.page_id.default = page.id
> db.document.page_id.writable = False
> grid = SQLFORM.grid(db.document.page_id==page.id, args=[page.id])
> return dict(page=page, grid=grid)
> 
> @auth.requires_login()
> def user():
> """
> exposes:
> http:///[app]/default/user/login
> http:///[app]/default/user/logout
> http:///[app]/default/user/register
> http:///[app]/default/user/profile
> http:///[app]/default/user/retrieve_password
> http:///[app]/default/user/change_password
> use @auth.requires_login()
> @auth.requires_membership('group name')
> @auth.requires_permission('read','table name',record_id)
> to decorate functions that need access control
> """
> return dict(form=auth())
>
> Here are the views:
> Index:
> {{left_sidebar_enabled,right_sidebar_enabled=False,True}}
> {{extend 'layout.html'}}
> Available Wiki Pages
> [ {{=A('search', _href=URL('search'))}} ]
> {{for page in pages:}}
>{{=LI(A(page.title, _href=URL('show', args=page.id)))}}
> {{pass}}
> [ {{=A('create page', _href=URL('create'))}}]
>
> Create:
> {{extend 'layout.html'}}
> Create new wiki page
> {{=form}}
>
> Show:
> {{extend 'layout.html'}}
> {{=page.title}}
> [ {{=A('edit', _href=URL('edit', args=request.args))}} ]
> [ {{=A('documents', _href=URL('documents', args=request.args))}} ]
> {{=MARKMIN(page.body)}}
> Comments
> {{for comment in comments:}}
>   {{=db.auth_user[comment.created_by].first_name}} on 
> {{=comment.created_on}} says {{=comment.body}}
> {{pass}}
> Post a comment
> {{=form}}
>
> Edit:
> {{extend 'layout.html'}}
> Edit wiki page
> [ {{=A('show', _href=URL('show', args=request.args))}} ] 
> {{=form}}
>
> documents:
> {{extend 'layout.html'}}
> Documents for page: {{=page.title}}
> [ {{=A('show', _href=URL('show', args=request.args))}} ]
> Doucments
> {{=grid}}
>
> Thanks
>
>
> On Thursday, May 10, 2012 6:09:09 AM UTC-5, Anthony wrote:
>>
>> Can you post your exact code (models, controllers, views)?
>>
>> On Thursday, May 10, 2012 1:22:14 AM UTC-4, man24 wrote:
>>>
>>> Sorry for reposting the question.  I am following the 'wiki' example 
>>> given in the web2py book.  After first implementation, it worked fine, 
>>> however, now when I click on any action that requires login or even 
>>> clicking the 'login' from the top menu, I get this response from my web 
>>> browser:  'too many redirects occurred when trying to open'.  Any idea what 
>>> could be wrong.  The 'wiki' app was w

[web2py] Re: too many redirect message when using login

[Anthony]

Solved, I 
think: https://groups.google.com/d/msg/web2py/EzSNrQt4SZk/ZggJrIqad5UJ

On Thursday, May 10, 2012 12:27:28 AM UTC-4, man24 wrote:
>
> I am new to web2py and has been following the online book to learn it.  I 
> was following the 'wiki' example and everything worked fine the first time, 
> however, today when I click on any action that requires login or even the 
> login link in the menu, web browser cannot open the page and gives 
> following message:  'too many redirects occurred trying to open the page'. 
>  I am using web2py on a mac.  Thanks for your help.

[web2py] Re: Manual image upload

[王康]

hi, have you guys solved this problem?

Because now I'm in the same problem >_<

On Sunday, December 26, 2010 4:15:54 AM UTC+8, mdipierro wrote:
>
> Email me your app. 
>
> On Dec 25, 2:08 pm, "Arun K.Rajeevan"  wrote: 
> > https://groups.google.com/d/msg/web2py/UtBmAdvY7E0/QQ5rBVM4lEIJ 
> > 
> > tables are given in that post. 
> > 
> > But, as I point out last, problem may be in conversion of file into blob 
> and 
> > getting it back. 
> > BTW, if you can suggest a solution discussed in above link that will be 
> > great; although this problem is amusing, I'm in need for a solution.

Re: [web2py] Need a little help in code review (a function eating up all memory)

[szimszon]

Could it be related?  
https://groups.google.com/forum/#!topic/web2py/hmsupVHdDHo/discussion (Memory 
leak in standalone DAL (issue #731), can you please help test?)

2012. május 10., csütörtök 22:32:53 UTC+2 időpontban szimszon a következőt 
írta:
>
> Okay. It's clear.
>
> I'm only puzzled about why the memory didn't get freed or reused after 
> execution is finished. And if I execute the controller function in 1-2min 
> interval mem is still not reused.
>
> So I understand it can eat up the memory but why is all memory locked 
> forever and didn't get reused - in my understanding (it's not much) in my 
> case GC do not free mem at all in python level. So some reference is still 
> intact after function is finished.
>
>
> 2012. május 10., csütörtök 21:53:06 UTC+2 időpontban Richard a következőt 
> írta:
>>
>> Ok, you don't need it to works all the time.
>>
>> Did you get it to update your records?
>>
>> If not, and if as you said it is a one trip, you can just treat the whole 
>> records batch by batch...
>>
>> Look here : 
>> http://web2py.com/books/default/chapter/29/14#Populating-database-with-dummy-data
>>
>> for i in range(10):
>>
>> populate(db.mytable,100)
>>
>> db.commit()
>>
>>
>> In this example only 100 records are populated at a time between 
>> db.commit()... So maybe you just have to wrap for loop that will 
>> db.commit() a couples of time during your processing.
>>
>> Richard
>>
>>
>> On Thu, May 10, 2012 at 3:32 PM, szimszon  wrote:
>>
>>> I had to store files and a lot of properties for it. It was in csv. But 
>>> after I processed it we figured out that not all value was correct in csv 
>>> but it was a bit redundant. So I can correct it by go through all the 
>>> records row by row. So that was a one time trip.
>>>
>>> I just realized after the process I had no memory left. So now I'm 
>>> investigating what happened... 
>>>
>>> 2012. május 10., csütörtök 21:00:05 UTC+2 időpontban Richard a 
>>> következőt írta:

 Yes but in this case it is not for the entire reecords...

 Why would you return a full list of all the records?

 I don't understand what is the purpose of listar that you return in the 
 view under a html table, why do you need to return all the 10+ entries?

 Richard

 On Thu, May 10, 2012 at 2:56 PM, szimszon  wrote:

> In book it is a recommended way to iterate over sql results:
>
> http://web2py.com/books/**default/chapter/29/6
>
> You can do all the steps in one statement:
>
> 1.
> 2.
> 3.
>
> >>> for row in db(db.person.name=='Alex').sel**ect():
>
>
> print row.name
>
> Alex
>
>
>
>
> 2012. május 10., csütörtök 20:42:22 UTC+2 időpontban Bruce Wade a 
> következőt írta:
>
>> Sorry, you really need to read more about how python works. If you 
>> learn how for loops work and memory you will understand the problem. 
>>
>> One solution do the query before the for loop then loop through the 
>> objects. This may help at bit. Research xrange vs range
>>
>>
>> On Thu, May 10, 2012 at 11:30 AM, szimszon wrote:
>>
>>> Sorry I don't understand. What do you mean "achieve with join"?
>>>
>>> There is an empty for loop with db.executesql() without join. And it 
>>> is eating up the memory. :(
>>>
>>> 2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a 
>>> következőt írta:
>>>
 You can't manage what you want to achieve with join?

 Richard

 On Thu, May 10, 2012 at 10:48 AM, szimszon wrote:

> Sorry for my dumbness but if something is wrong with my code 
> please point me the right line. I'm not so good in English if it 
> comes to 
> "object instance count" and so. Yeah I know I should go and do some 
> milkmaid job :) but I'm curious.
>
> I'm just define some variable:
>
> lista = list()
> last_row = None
> next_page_number = 0
>
> Go in a for loop that just assign the db query result one-by-one 
> to row variable.
>
> After that I assign the TABLE() helper to a list variable. That is 
> it.
>
>
>
> 2012. május 10., csütörtök 16:34:49 UTC+2 időpontban Bruce Wade a 
> következőt írta:
>
>> Using direct sql query or DAL is going to cause the exact same 
>> problem in this situation.
>>
>> On Thu, May 10, 2012 at 7:28 AM, szimszon wrote:
>>
>>> It's postgres:// in a 
>>> Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and
>>>
>>> Python 2.7.3 (default, Apr 20 2012, 22:44:07) 
>>> [GCC 4.6.3] on linux2
>>>
>>> python-psycopg2 2.4.5-1
>>>
>>>
>>> 2012. máj

[web2py] Re: Vertical lines around database string values

[Yarin]

This was because I accidentally had trailing commas on some insert param 
statements, causing them to be tuples instead of strings. My bad.

On Sunday, April 15, 2012 9:41:28 AM UTC-4, Yarin wrote:
>
> This went away, don't know why or how..
>
> On Saturday, April 14, 2012 11:17:51 AM UTC-4, Yarin wrote:
>>
>> When inserting records with web2py's db class, my string values have 
>> vertical bars surrounding them. I don't remember seeing this behavior 
>> before. What's more, if i insert records through web2py's db admin 
>> interface, i don't get vertical bars.
>>
>> What are the vertical bars - is it a unicode thing? - and how are we 
>> supposed to handle them?
>>
>

Re: [web2py] Re: Nginx-uwsgi problem.

[Michele Comitini]

The high load on web2py nodes seems to point to code in web2py.  If it
were a problem with postgres you would have a high load on postgresql
and a lot of wait state and little CPU time resulting in little uptime
on web2py nodes but long page rendering times.
I suggest to try to convert some logic to use raw resultsets using
executesql instead of DAL Row objects.  But before doing that try the
query on postgres directly: you can use the _select() method to obtain
the query generated by the DAL.  If postgresql answers slowly try
adding indexes on columns as requested by EXPLAIN.
If postgresql answers fast try the guilty query with the DAL in a
python shell (i.e. python web2py.py -M -S ). If it slow than
you have found the cause.

Else keep using top to find if other processes are infesting the CPU
maybe it is a simple problem of "ping pong" or swappiness.  Simple
tuning of uWSGI could suffice.  As a rule of thumb you should not have
the number of web2py processes be more than twice the number of cores.

mic


2012/5/10 Bruce Wade :
> Web2py is on 3 different servers/nodes, postgresql is on it's own node with
> 8GB ram.
>
> CPU is being used by uwsgi so web2py. The slowness I think is from DB
> queries as when you load a page without the DB involved much it loads
> quickly
>
> The serving ads part is not a problem it is the other pages on the website.
> At least not the adviewer the banner ads are new. The adviewer has served
> over 29 million ads.
>
> I will try disabling the banner ads for now and set them so they are
> querying from a completely different server, maybe using mongodb and node.js
>
>
> On Thu, May 10, 2012 at 11:28 AM, pbreit  wrote:
>>
>> Is your traffic from serving ads or users coming to your web site? Have
>> you exhausted caching opportunities?
>
>
>
>
> --
> --
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>

Re: [web2py] Re: Nginx-uwsgi problem.

[Bruce Wade]

Yes there are a LOT of wait state on the web2py nodes and high CPU

I will try your suggestions.

Thanks,
Bruce

On Thu, May 10, 2012 at 2:10 PM, Michele Comitini <
michele.comit...@gmail.com> wrote:

> The high load on web2py nodes seems to point to code in web2py.  If it
> were a problem with postgres you would have a high load on postgresql
> and a lot of wait state and little CPU time resulting in little uptime
> on web2py nodes but long page rendering times.
> I suggest to try to convert some logic to use raw resultsets using
> executesql instead of DAL Row objects.  But before doing that try the
> query on postgres directly: you can use the _select() method to obtain
> the query generated by the DAL.  If postgresql answers slowly try
> adding indexes on columns as requested by EXPLAIN.
> If postgresql answers fast try the guilty query with the DAL in a
> python shell (i.e. python web2py.py -M -S ). If it slow than
> you have found the cause.
>
> Else keep using top to find if other processes are infesting the CPU
> maybe it is a simple problem of "ping pong" or swappiness.  Simple
> tuning of uWSGI could suffice.  As a rule of thumb you should not have
> the number of web2py processes be more than twice the number of cores.
>
> mic
>
>
> 2012/5/10 Bruce Wade :
> > Web2py is on 3 different servers/nodes, postgresql is on it's own node
> with
> > 8GB ram.
> >
> > CPU is being used by uwsgi so web2py. The slowness I think is from DB
> > queries as when you load a page without the DB involved much it loads
> > quickly
> >
> > The serving ads part is not a problem it is the other pages on the
> website.
> > At least not the adviewer the banner ads are new. The adviewer has served
> > over 29 million ads.
> >
> > I will try disabling the banner ads for now and set them so they are
> > querying from a completely different server, maybe using mongodb and
> node.js
> >
> >
> > On Thu, May 10, 2012 at 11:28 AM, pbreit  wrote:
> >>
> >> Is your traffic from serving ads or users coming to your web site? Have
> >> you exhausted caching opportunities?
> >
> >
> >
> >
> > --
> > --
> > Regards,
> > Bruce Wade
> > http://ca.linkedin.com/in/brucelwade
> > http://www.wadecybertech.com
> > http://www.fittraineronline.com - Fitness Personal Trainers Online
> > http://www.warplydesigned.com
> >
>



-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

Re: [web2py] why so many instances of console-kit-daemon?

[Michele Comitini]

That is for managing the consoles nothing to do with web2py.  IIRC
there was a bug with console-kit eating too many resources, try to see
if updating it works.

mic


2012/5/10 Carlos :
> Hi,
>
> My web2py app runs on ubuntu server 10.04, with postgresql, nginx,
> uwsgi-python.
>
> With htop, I notice there are 64 instances of the following (each with MEM
> 0.3%):
>
>    console-kit-daemon --no-daemon
>
> Does anybody know what's that about?.
>
> Does this mean they are using almost 20% of memory (0.3% x 64 instances)?.
>
> Thanks,
>
>    Carlos
>

[web2py] Re: MongoDB Adapter error in select

[Francisco Costa]

Just add a '\' at the end of line 4664 and your are done!

Great job Massimo!


On Thursday, May 10, 2012 9:12:48 PM UTC+1, Massimo Di Pierro wrote:
>
> This helps. I have another attempt to fix this in trunk. Hard to test it 
> since I do not have mongodb installed.
>
> On Thursday, 10 May 2012 08:39:22 UTC-5, Francisco Costa wrote:
>>
>> Still not working. I think Fields order is wrong.
>>
>> At the end of the select() function I write this:
>>
>> a = []
>> for f in fields:
>> a.append(f.type)
>> return dict(rows=rows, fields=a, colnames=colnames)
>>
>> and I got this:
>>
>> colnames:
>> city
>> age
>> _id
>> name
>> fields:
>> id
>> text
>> integer
>> string
>> rows:
>> Toronto
>> 66L
>> 24652490551171733682233802752L
>> John
>>
>>
>>

Re: [web2py] Need a little help in code review (a function eating up all memory)

[Martín Mulone]

The problem is fetchall(), is getting all the records to memory.
If you want to iterate a large dataset or not so heavy, but have a table
with many fields, perhaps you can do it by blocks.

For example using raw sql with dal: example: this fetch by 1000 records

queryraw = "select * from mytable %(limit_query)s"
def records(query):
rows = db.executesql(query)
if rows:
return True
else:
return False

rec = 0
stage = 0
block = 1000while True:
offset = stage * block
query = queryraw % {'limitquery': 'LIMIT %s, %s' % (offset, block)}
if not records(query):
break

rows = db.executesql(query)
for row in rows:
rec += 1
# do something

stage += 1

This is not fast but keep memory low.


2012/5/10 szimszon 

> Could it be related?
> https://groups.google.com/forum/#!topic/web2py/hmsupVHdDHo/discussion (Memory
> leak in standalone DAL (issue #731), can you please help test?)
>
> 2012. május 10., csütörtök 22:32:53 UTC+2 időpontban szimszon a következőt
> írta:
>
>> Okay. It's clear.
>>
>> I'm only puzzled about why the memory didn't get freed or reused after
>> execution is finished. And if I execute the controller function in 1-2min
>> interval mem is still not reused.
>>
>> So I understand it can eat up the memory but why is all memory locked
>> forever and didn't get reused - in my understanding (it's not much) in my
>> case GC do not free mem at all in python level. So some reference is still
>> intact after function is finished.
>>
>>
>> 2012. május 10., csütörtök 21:53:06 UTC+2 időpontban Richard a következőt
>> írta:
>>>
>>> Ok, you don't need it to works all the time.
>>>
>>> Did you get it to update your records?
>>>
>>> If not, and if as you said it is a one trip, you can just treat the
>>> whole records batch by batch...
>>>
>>> Look here : http://web2py.com/books/**default/chapter/29/14#**
>>> Populating-database-with-**dummy-data
>>>
>>> for i in range(10):
>>>
>>> populate(db.mytable,100)
>>>
>>> db.commit()
>>>
>>>
>>> In this example only 100 records are populated at a time between
>>> db.commit()... So maybe you just have to wrap for loop that will
>>> db.commit() a couples of time during your processing.
>>>
>>> Richard
>>>
>>>
>>> On Thu, May 10, 2012 at 3:32 PM, szimszon  wrote:
>>>
 I had to store files and a lot of properties for it. It was in csv. But
 after I processed it we figured out that not all value was correct in csv
 but it was a bit redundant. So I can correct it by go through all the
 records row by row. So that was a one time trip.

 I just realized after the process I had no memory left. So now I'm
 investigating what happened...

 2012. május 10., csütörtök 21:00:05 UTC+2 időpontban Richard a
 következőt írta:
>
> Yes but in this case it is not for the entire reecords...
>
> Why would you return a full list of all the records?
>
> I don't understand what is the purpose of listar that you return in
> the view under a html table, why do you need to return all the 10+
> entries?
>
> Richard
>
> On Thu, May 10, 2012 at 2:56 PM, szimszon  wrote:
>
>> In book it is a recommended way to iterate over sql results:
>>
>> http://web2py.com/books/**defaul**t/chapter/29/6
>>
>> You can do all the steps in one statement:
>>
>> 1.
>> 2.
>> 3.
>>
>> >>> for row in db(db.person.name=='Alex').select():
>>
>>
>> print row.name
>>
>> Alex
>>
>>
>>
>>
>> 2012. május 10., csütörtök 20:42:22 UTC+2 időpontban Bruce Wade a
>> következőt írta:
>>
>>> Sorry, you really need to read more about how python works. If you
>>> learn how for loops work and memory you will understand the problem.
>>>
>>> One solution do the query before the for loop then loop through the
>>> objects. This may help at bit. Research xrange vs range
>>>
>>>
>>> On Thu, May 10, 2012 at 11:30 AM, szimszon wrote:
>>>
 Sorry I don't understand. What do you mean "achieve with join"?

 There is an empty for loop with db.executesql() without join. And
 it is eating up the memory. :(

 2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a
 következőt írta:

> You can't manage what you want to achieve with join?
>
> Richard
>
> On Thu, May 10, 2012 at 10:48 AM, szimszon wrote:
>
>> Sorry for my dumbness but if something is wrong with my code
>> please point me the right line. I'm not so good in English if it 
>> comes to
>> "object instance count" and so. Yeah I know I should go and do some
>> milkmaid job :) but I'm curious.
>>
>>>

[web2py] Re: MongoDB Adapter error in select

[Francisco Costa]

I found another problem!!

if you do this:
mongo.user.insert(name='Jim',  age=18, city='Detroit')
mongo.user.insert(name='Jack', age=18)

and then select
users = mongo(mongo.user.age==18).select()

you don't have the field "city" in one of the rows, so you get this error:

users = mongo(mongo.user.age==18).select()
  File "/opt/web2py/gluon/dal.py", line 8123, in select
return adapter.select(self.query,fields,attributes)
  File "/opt/web2py/gluon/dal.py", line 4700, in select
return processor(rows,fields,colnames,False)
  File "/opt/web2py/gluon/dal.py", line 1681, in parse
self.parse_value(value, fields[j].type,blob_decode)
  File "/opt/web2py/gluon/dal.py", line 1575, in parse_value
return self.parsemap[key](value,field_type)
  File "/opt/web2py/gluon/dal.py", line 1641, in parse_id
return int(value)
ValueError: invalid literal for int() with base 10: 'Jack'



On Thursday, May 10, 2012 10:16:32 PM UTC+1, Francisco Costa wrote:
>
> Just add a '\' at the end of line 4664 and your are done!
>
> Great job Massimo!
>
>
> On Thursday, May 10, 2012 9:12:48 PM UTC+1, Massimo Di Pierro wrote:
>>
>> This helps. I have another attempt to fix this in trunk. Hard to test it 
>> since I do not have mongodb installed.
>>
>> On Thursday, 10 May 2012 08:39:22 UTC-5, Francisco Costa wrote:
>>>
>>> Still not working. I think Fields order is wrong.
>>>
>>> At the end of the select() function I write this:
>>>
>>> a = []
>>> for f in fields:
>>> a.append(f.type)
>>> return dict(rows=rows, fields=a, colnames=colnames)
>>>
>>> and I got this:
>>>
>>> colnames:
>>> city
>>> age
>>> _id
>>> name
>>> fields:
>>> id
>>> text
>>> integer
>>> string
>>> rows:
>>> Toronto
>>> 66L
>>> 24652490551171733682233802752L
>>> John
>>>
>>>
>>>

[web2py] Setup web2py in virtualenv

[mrtn]

I have installed virtualenv and setup an environment for a web2py project. 
I came across a post before where people say you can use pip to install 
web2py, but it is still experimental and possibly not working well. I 
wonder if I can just copy the whole web2py source folder to somewhere 
(/bin, or, /lib) inside the environment directory, and run python 
web2py.pyevery time there? Would that work?

[web2py] Re: MongoDB Adapter error in select

[Massimo Di Pierro]

What's the model?

On Thursday, 10 May 2012 16:27:51 UTC-5, Francisco Costa wrote:
>
> I found another problem!!
>
> if you do this:
> mongo.user.insert(name='Jim',  age=18, city='Detroit')
> mongo.user.insert(name='Jack', age=18)
>
> and then select
> users = mongo(mongo.user.age==18).select()
>
> you don't have the field "city" in one of the rows, so you get this error:
>
> users = mongo(mongo.user.age==18).select()
>   File "/opt/web2py/gluon/dal.py", line 8123, in select
> return adapter.select(self.query,fields,attributes)
>   File "/opt/web2py/gluon/dal.py", line 4700, in select
> return processor(rows,fields,colnames,False)
>   File "/opt/web2py/gluon/dal.py", line 1681, in parse
> self.parse_value(value, fields[j].type,blob_decode)
>   File "/opt/web2py/gluon/dal.py", line 1575, in parse_value
> return self.parsemap[key](value,field_type)
>   File "/opt/web2py/gluon/dal.py", line 1641, in parse_id
> return int(value)
> ValueError: invalid literal for int() with base 10: 'Jack'
>
>
>
> On Thursday, May 10, 2012 10:16:32 PM UTC+1, Francisco Costa wrote:
>>
>> Just add a '\' at the end of line 4664 and your are done!
>>
>> Great job Massimo!
>>
>>
>> On Thursday, May 10, 2012 9:12:48 PM UTC+1, Massimo Di Pierro wrote:
>>>
>>> This helps. I have another attempt to fix this in trunk. Hard to test it 
>>> since I do not have mongodb installed.
>>>
>>> On Thursday, 10 May 2012 08:39:22 UTC-5, Francisco Costa wrote:

 Still not working. I think Fields order is wrong.

 At the end of the select() function I write this:

 a = []
 for f in fields:
 a.append(f.type)
 return dict(rows=rows, fields=a, colnames=colnames)

 and I got this:

 colnames:
 city
 age
 _id
 name
 fields:
 id
 text
 integer
 string
 rows:
 Toronto
 66L
 24652490551171733682233802752L
 John

[web2py] Re: MongoDB Adapter error in select

[Francisco Costa]

i didn't understand your question..
This is the complete code

import sys
import time
from gluon.dal import DAL, Field
mongo = DAL('mongodb://localhost:27017/sapo')
mongo.define_table('user',
 Field('name', 'text'),
 Field('age',  'integer'),
 Field('city', 'string')
 )

def insert_users():
mongo.user.insert(name='John', age=66, city='Toronto')
mongo.user.insert(name='Mark', age=43, city='Boston')
mongo.user.insert(name='Tom',  age=43, city='Detroit')
mongo.user.insert(name='Jim',  age=18, city='Detroit')
mongo.user.insert(name='Jack', age=18)
mongo.user.insert(name='Eric', city='Boston')
return 'users in database'

def find_users():
users = mongo(mongo.user.age==18).select()
return dict(users=users)




On Thursday, May 10, 2012 11:58:36 PM UTC+1, Massimo Di Pierro wrote:
>
> What's the model?
>
> On Thursday, 10 May 2012 16:27:51 UTC-5, Francisco Costa wrote:
>>
>> I found another problem!!
>>
>> if you do this:
>> mongo.user.insert(name='Jim',  age=18, city='Detroit')
>> mongo.user.insert(name='Jack', age=18)
>>
>> and then select
>> users = mongo(mongo.user.age==18).select()
>>
>> you don't have the field "city" in one of the rows, so you get this error:
>>
>> users = mongo(mongo.user.age==18).select()
>>   File "/opt/web2py/gluon/dal.py", line 8123, in select
>> return adapter.select(self.query,fields,attributes)
>>   File "/opt/web2py/gluon/dal.py", line 4700, in select
>> return processor(rows,fields,colnames,False)
>>   File "/opt/web2py/gluon/dal.py", line 1681, in parse
>> self.parse_value(value, fields[j].type,blob_decode)
>>   File "/opt/web2py/gluon/dal.py", line 1575, in parse_value
>> return self.parsemap[key](value,field_type)
>>   File "/opt/web2py/gluon/dal.py", line 1641, in parse_id
>> return int(value)
>> ValueError: invalid literal for int() with base 10: 'Jack'
>>
>>
>>
>> On Thursday, May 10, 2012 10:16:32 PM UTC+1, Francisco Costa wrote:
>>>
>>> Just add a '\' at the end of line 4664 and your are done!
>>>
>>> Great job Massimo!
>>>
>>>
>>> On Thursday, May 10, 2012 9:12:48 PM UTC+1, Massimo Di Pierro wrote:

 This helps. I have another attempt to fix this in trunk. Hard to test 
 it since I do not have mongodb installed.

 On Thursday, 10 May 2012 08:39:22 UTC-5, Francisco Costa wrote:
>
> Still not working. I think Fields order is wrong.
>
> At the end of the select() function I write this:
>
> a = []
> for f in fields:
> a.append(f.type)
> return dict(rows=rows, fields=a, colnames=colnames)
>
> and I got this:
>
> colnames:
> city
> age
> _id
> name
> fields:
> id
> text
> integer
> string
> rows:
> Toronto
> 66L
> 24652490551171733682233802752L
> John
>
>
>

[web2py] Load data from sqlite file?

[Dave]

Any examples or suggestions on how to load data from a user-uploaded 
.sqlite database?

I will have the database structure of the uploaded files ahead of time 
(they are being generated by a mobile app), so I need a way to transfer the 
data from an uploaded .sqlite file to my web2py tables.  is it possible to 
just use the uploaded file as a new dal, then transfer the data to the main 
dal, all within a function?

Thanks in advance,
Dave

[web2py] Re: MongoDB Adapter error in select

[Massimo Di Pierro]

Now I understand better where all these problems come from. I shall fix it 
tonight.

On Thursday, 10 May 2012 18:02:24 UTC-5, Francisco Costa wrote:
>
> i didn't understand your question..
> This is the complete code
>
> import sys
> import time
> from gluon.dal import DAL, Field
> mongo = DAL('mongodb://localhost:27017/sapo')
> mongo.define_table('user',
>  Field('name', 'text'),
>  Field('age',  'integer'),
>  Field('city', 'string')
>  )
>
> def insert_users():
> mongo.user.insert(name='John', age=66, city='Toronto')
> mongo.user.insert(name='Mark', age=43, city='Boston')
> mongo.user.insert(name='Tom',  age=43, city='Detroit')
> mongo.user.insert(name='Jim',  age=18, city='Detroit')
> mongo.user.insert(name='Jack', age=18)
> mongo.user.insert(name='Eric', city='Boston')
> return 'users in database'
>
> def find_users():
> users = mongo(mongo.user.age==18).select()
> return dict(users=users)
>
>
>
>
> On Thursday, May 10, 2012 11:58:36 PM UTC+1, Massimo Di Pierro wrote:
>>
>> What's the model?
>>
>> On Thursday, 10 May 2012 16:27:51 UTC-5, Francisco Costa wrote:
>>>
>>> I found another problem!!
>>>
>>> if you do this:
>>> mongo.user.insert(name='Jim',  age=18, city='Detroit')
>>> mongo.user.insert(name='Jack', age=18)
>>>
>>> and then select
>>> users = mongo(mongo.user.age==18).select()
>>>
>>> you don't have the field "city" in one of the rows, so you get this 
>>> error:
>>>
>>> users = mongo(mongo.user.age==18).select()
>>>   File "/opt/web2py/gluon/dal.py", line 8123, in select
>>> return adapter.select(self.query,fields,attributes)
>>>   File "/opt/web2py/gluon/dal.py", line 4700, in select
>>> return processor(rows,fields,colnames,False)
>>>   File "/opt/web2py/gluon/dal.py", line 1681, in parse
>>> self.parse_value(value, fields[j].type,blob_decode)
>>>   File "/opt/web2py/gluon/dal.py", line 1575, in parse_value
>>> return self.parsemap[key](value,field_type)
>>>   File "/opt/web2py/gluon/dal.py", line 1641, in parse_id
>>> return int(value)
>>> ValueError: invalid literal for int() with base 10: 'Jack'
>>>
>>>
>>>
>>> On Thursday, May 10, 2012 10:16:32 PM UTC+1, Francisco Costa wrote:

 Just add a '\' at the end of line 4664 and your are done!

 Great job Massimo!


 On Thursday, May 10, 2012 9:12:48 PM UTC+1, Massimo Di Pierro wrote:
>
> This helps. I have another attempt to fix this in trunk. Hard to test 
> it since I do not have mongodb installed.
>
> On Thursday, 10 May 2012 08:39:22 UTC-5, Francisco Costa wrote:
>>
>> Still not working. I think Fields order is wrong.
>>
>> At the end of the select() function I write this:
>>
>> a = []
>> for f in fields:
>> a.append(f.type)
>> return dict(rows=rows, fields=a, colnames=colnames)
>>
>> and I got this:
>>
>> colnames:
>> city
>> age
>> _id
>> name
>> fields:
>> id
>> text
>> integer
>> string
>> rows:
>> Toronto
>> 66L
>> 24652490551171733682233802752L
>> John
>>
>>
>>