I had to store files and a lot of properties for it. It was in csv. But after I processed it we figured out that not all value was correct in csv but it was a bit redundant. So I can correct it by go through all the records row by row. So that was a one time trip.
I just realized after the process I had no memory left. So now I'm investigating what happened... 2012. május 10., csütörtök 21:00:05 UTC+2 időpontban Richard a következőt írta: > > Yes but in this case it is not for the entire reecords... > > Why would you return a full list of all the records? > > I don't understand what is the purpose of listar that you return in the > view under a html table, why do you need to return all the 100000+ entries? > > Richard > > On Thu, May 10, 2012 at 2:56 PM, szimszon <szims...@gmail.com> wrote: > >> In book it is a recommended way to iterate over sql results: >> >> http://web2py.com/books/default/chapter/29/6 >> >> You can do all the steps in one statement: >> >> 1. >> 2. >> 3. >> >> >>> for row in db(db.person.name=='Alex').select(): >> >> print row.name >> Alex >> >> >> >> >> 2012. május 10., csütörtök 20:42:22 UTC+2 időpontban Bruce Wade a >> következőt írta: >> >>> Sorry, you really need to read more about how python works. If you learn >>> how for loops work and memory you will understand the problem. >>> >>> One solution do the query before the for loop then loop through the >>> objects. This may help at bit. Research xrange vs range >>> >>> >>> On Thu, May 10, 2012 at 11:30 AM, szimszon <szims...@gmail.com> wrote: >>> >>>> Sorry I don't understand. What do you mean "achieve with join"? >>>> >>>> There is an empty for loop with db.executesql() without join. And it is >>>> eating up the memory. :( >>>> >>>> 2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a >>>> következőt írta: >>>> >>>>> You can't manage what you want to achieve with join? >>>>> >>>>> Richard >>>>> >>>>> On Thu, May 10, 2012 at 10:48 AM, szimszon <szims...@gmail.com> wrote: >>>>> >>>>>> Sorry for my dumbness but if something is wrong with my code please >>>>>> point me the right line. I'm not so good in English if it comes to >>>>>> "object >>>>>> instance count" and so. Yeah I know I should go and do some milkmaid job >>>>>> :) >>>>>> but I'm curious. >>>>>> >>>>>> I'm just define some variable: >>>>>> >>>>>> lista = list() >>>>>> last_row = None >>>>>> next_page_number = 0 >>>>>> >>>>>> Go in a for loop that just assign the db query result one-by-one to >>>>>> row variable. >>>>>> >>>>>> After that I assign the TABLE() helper to a list variable. That is it. >>>>>> >>>>>> >>>>>> >>>>>> 2012. május 10., csütörtök 16:34:49 UTC+2 időpontban Bruce Wade a >>>>>> következőt írta: >>>>>> >>>>>>> Using direct sql query or DAL is going to cause the exact same >>>>>>> problem in this situation. >>>>>>> >>>>>>> On Thu, May 10, 2012 at 7:28 AM, szimszon <szims...@gmail.com>wrote: >>>>>>> >>>>>>>> It's postgres:// in a >>>>>>>> Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and >>>>>>>> >>>>>>>> Python 2.7.3 (default, Apr 20 2012, 22:44:07) >>>>>>>> [GCC 4.6.3] on linux2 >>>>>>>> >>>>>>>> python-psycopg2 2.4.5-1 >>>>>>>> >>>>>>>> >>>>>>>> 2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a >>>>>>>> következőt írta: >>>>>>>> >>>>>>>>> Just for curiosity, what happens if you do it in pure sql? >>>>>>>>> >>>>>>>>> >>>>>>>>> for row in db.executesql("**sele******ct * from file_properties >>>>>>>>> where id > 0"): >>>>>>>>> >>>>>>>>> # do something >>>>>>>>> >>>>>>>>> >>>>>>>>> Does it have a lower memory usage? >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, May 10, 2012 at 4:14 AM, Bruce Wade <<bruce.w...@gmail.com> >>>>>>>>> > wrote: >>>>>>>>> >>>>>>>>>> for row in db( db.file_**propert******ies.id > 0 ).select( >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Bruno Rocha >>>>>>>>> [http://rochacbruno.com.br] >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> -- >>>>>>> Regards, >>>>>>> Bruce Wade >>>>>>> http://ca.linkedin.com/in/**bruc****elwade<http://ca.linkedin.com/in/brucelwade> >>>>>>> http://www.wadecybertech.com >>>>>>> http://www.fittraineronline.**co****m<http://www.fittraineronline.com>- >>>>>>> Fitness Personal Trainers Online >>>>>>> http://www.warplydesigned.com >>>>>>> >>>>>>> >>>>> >>> >>> >>> -- >>> -- >>> Regards, >>> Bruce Wade >>> http://ca.linkedin.com/in/**brucelwade<http://ca.linkedin.com/in/brucelwade> >>> http://www.wadecybertech.com >>> http://www.fittraineronline.**com <http://www.fittraineronline.com> - >>> Fitness Personal Trainers Online >>> http://www.warplydesigned.com >>> >>> >
