Re: Finding size of Variable

2014-02-12 Thread Rustom Mody
On Wednesday, February 12, 2014 7:55:32 PM UTC+5:30, Mark Lawrence wrote: > On 12/02/2014 14:14, Rustom Mody wrote: > > On Wednesday, February 12, 2014 7:34:42 PM UTC+5:30, Mark Lawrence wrote: > >> I ask you, members of the jury, to find the accused, jmf, guilty of > >> writing nonsense and delibe

Re: Finding size of Variable

2014-02-12 Thread Mark Lawrence
On 12/02/2014 14:14, Rustom Mody wrote: On Wednesday, February 12, 2014 7:34:42 PM UTC+5:30, Mark Lawrence wrote: I ask you, members of the jury, to find the accused, jmf, guilty of writing nonsense and deliberately using google groups to double line space. The evidence is directly above and q

Re: Finding size of Variable

2014-02-12 Thread Rustom Mody
On Wednesday, February 12, 2014 7:34:42 PM UTC+5:30, Mark Lawrence wrote: > I ask you, members of the jury, to find the accused, jmf, guilty of > writing nonsense and deliberately using google groups to double line > space. The evidence is directly above and quite clearly prooves, beyond > a r

Re: Finding size of Variable

2014-02-12 Thread Mark Lawrence
On 12/02/2014 07:49, wxjmfa...@gmail.com wrote: Le mardi 11 février 2014 20:04:02 UTC+1, Mark Lawrence a écrit : On 11/02/2014 18:53, wxjmfa...@gmail.com wrote: Le lundi 10 février 2014 15:43:08 UTC+1, Tim Chase a écrit : On 2014-02-10 06:07, wxjmfa...@gmail.com wrote: Python does no

Re: Finding size of Variable

2014-02-12 Thread Jussi Piitulainen
Chris Angelico writes: > On Wed, Feb 12, 2014 at 7:57 PM, Jussi Piitulainen wrote: > >> In Python, integers have arbitrary precision, but floats, Fractions, > >> and Decimals, don't. Nearly any operation on arbitrarily large > >> numbers will be either more accurate or more efficient (maybe both) >

Re: Finding size of Variable

2014-02-12 Thread Chris Angelico
On Wed, Feb 12, 2014 at 7:57 PM, Jussi Piitulainen wrote: >> In Python, integers have arbitrary precision, but floats, Fractions, >> and Decimals, don't. Nearly any operation on arbitrarily large >> numbers will be either more accurate or more efficient (maybe both) >> with integers than with any

Re: Finding size of Variable

2014-02-12 Thread Jussi Piitulainen
Chris Angelico writes: > On Wed, Feb 12, 2014 at 6:49 PM, wrote: > > The day you find an operator working on the set of > > reals (R) and it is somehow "optimized" for N > > (the subset of natural numbers), let me know. ... > In Python, integers have arbitrary precision, but floats, Fractions,

Re: Finding size of Variable

2014-02-12 Thread Chris Angelico
On Wed, Feb 12, 2014 at 6:49 PM, wrote: > The day you find an operator working on the set of > reals (R) and it is somehow "optimized" for N > (the subset of natural numbers), let me know. I have yet to find any computer that works with the set of real numbers in any way. Never mind optimization

Re: Finding size of Variable

2014-02-11 Thread wxjmfauth
Le mardi 11 février 2014 20:04:02 UTC+1, Mark Lawrence a écrit : > On 11/02/2014 18:53, wxjmfa...@gmail.com wrote: > > > Le lundi 10 février 2014 15:43:08 UTC+1, Tim Chase a écrit : > > >> On 2014-02-10 06:07, wxjmfa...@gmail.com wrote: > > >> > > >>> Python does not save memory at all. A str (

Re: Finding size of Variable

2014-02-11 Thread Mark Lawrence
On 11/02/2014 18:53, wxjmfa...@gmail.com wrote: Le lundi 10 février 2014 15:43:08 UTC+1, Tim Chase a écrit : On 2014-02-10 06:07, wxjmfa...@gmail.com wrote: Python does not save memory at all. A str (unicode string) uses less memory only - and only - because and when one uses explicitly

Re: Finding size of Variable

2014-02-11 Thread wxjmfauth
Le lundi 10 février 2014 15:43:08 UTC+1, Tim Chase a écrit : > On 2014-02-10 06:07, wxjmfa...@gmail.com wrote: > > > Python does not save memory at all. A str (unicode string) > > > uses less memory only - and only - because and when one uses > > > explicitly characters which are consuming less

Re: Finding size of Variable

2014-02-11 Thread Neil Cerutti
On 2014-02-10, Ned Batchelder wrote: > On 2/10/14 9:43 AM, Tim Chase wrote: >>> The opposite of what the utf8/utf16 do! >>> >> sys.getsizeof(('a' * 100 + 'oe' + >> '\U0001').encode('utf-8')) >>> 123 >> sys.getsizeof(('a' * 100 + 'oe' + >> '\U0001').encode('utf-1

Re: Finding size of Variable

2014-02-10 Thread Ned Batchelder
On 2/10/14 9:43 AM, Tim Chase wrote: On 2014-02-10 06:07, wxjmfa...@gmail.com wrote: Python does not save memory at all. A str (unicode string) uses less memory only - and only - because and when one uses explicitly characters which are consuming less memory. Not only the memory gain is zero, P

Re: Finding size of Variable

2014-02-10 Thread Tim Chase
On 2014-02-10 06:07, wxjmfa...@gmail.com wrote: > Python does not save memory at all. A str (unicode string) > uses less memory only - and only - because and when one uses > explicitly characters which are consuming less memory. > > Not only the memory gain is zero, Python falls back to the > wors

Re: Finding size of Variable

2014-02-10 Thread Mark Lawrence
On 10/02/2014 14:25, Asaf Las wrote: On Monday, February 10, 2014 4:07:14 PM UTC+2, wxjm...@gmail.com wrote: Interesting sys.getsizeof('a' * 100) here you get string type sys.getsizeof(('a' * 100 + 'oe' + '\U0001').encode('utf-8')) and here bytes type ('a' * 1) type(('a

Re: Finding size of Variable

2014-02-10 Thread Asaf Las
On Monday, February 10, 2014 4:07:14 PM UTC+2, wxjm...@gmail.com wrote: Interesting > >>> sys.getsizeof('a' * 100) here you get string type > >>> sys.getsizeof(('a' * 100 + 'oe' + '\U0001').encode('utf-8')) and here bytes >>> type ('a' * 1) >>> type(('a' * 100 + 'oe' + '\U

Re: Finding size of Variable

2014-02-10 Thread wxjmfauth
Le samedi 8 février 2014 03:48:12 UTC+1, Steven D'Aprano a écrit : > > > We consider it A GOOD THING that Python spends memory for programmer > > convenience and safety. Python looks for memory optimizations when it can > > save large amounts of memory, not utterly trivial amounts. So in a Py

Re: Finding size of Variable

2014-02-08 Thread Rustom Mody
On Sunday, February 9, 2014 8:46:50 AM UTC+5:30, Ned Batchelder wrote: > On 2/8/14 10:09 PM, David Hutto wrote: > > Maybe I'll just roll my fat, bald, troll arse out from under the bridge, > > and comment back, off list, next time. > I'm not sure what happened in this thread. It might be that you

Re: Finding size of Variable

2014-02-08 Thread Ned Batchelder
On 2/8/14 10:09 PM, David Hutto wrote: Maybe I'll just roll my fat, bald, troll arse out from under the bridge, and comment back, off list, next time. I'm not sure what happened in this thread. It might be that you think Rustom Mody was referring to you when he said, "BTW: In my book this

Re: Finding size of Variable

2014-02-08 Thread David Hutto
Maybe I'll just roll my fat, bald, troll arse out from under the bridge, and comment back, off list, next time. -- https://mail.python.org/mailman/listinfo/python-list

Re: Finding size of Variable

2014-02-08 Thread Ned Batchelder
On 2/8/14 9:56 PM, David Hutto wrote: On Sat, Feb 8, 2014 at 8:25 PM, Rustom Mody mailto:rustompm...@gmail.com>> wrote: On Sunday, February 9, 2014 4:15:50 AM UTC+5:30, David Hutto wrote: > One could argue that if you're parsing a particular file, a very large one, that those 9 b

Re: Finding size of Variable

2014-02-08 Thread David Hutto
On Sat, Feb 8, 2014 at 9:59 PM, Chris Angelico wrote: > On Sun, Feb 9, 2014 at 1:56 PM, David Hutto > wrote: > >> > >> Yes... There are cases when python is an inappropriate language to > use... > >> So??? > > > > > > I didn't say she couldn't optimize in another language, and was just > > prot

Re: Finding size of Variable

2014-02-08 Thread Chris Angelico
On Sun, Feb 9, 2014 at 1:56 PM, David Hutto wrote: >> >> Yes... There are cases when python is an inappropriate language to use... >> So??? > > > I didn't say she couldn't optimize in another language, and was just > prototyping in Python. I just said she was optimizing her python > code...dufus.

Re: Finding size of Variable

2014-02-08 Thread David Hutto
On Sat, Feb 8, 2014 at 8:25 PM, Rustom Mody wrote: > On Sunday, February 9, 2014 4:15:50 AM UTC+5:30, David Hutto wrote: > > One could argue that if you're parsing a particular file, a very large > one, that those 9 bytes can go into the optimization of parsing > aforementioned file. Of, course w

Re: Finding size of Variable

2014-02-08 Thread Rustom Mody
On Sunday, February 9, 2014 4:15:50 AM UTC+5:30, David Hutto wrote: > One could argue that if you're parsing a particular file, a very large one, > that those 9 bytes can go into the optimization of parsing aforementioned > file. Of, course we have faster processors, so why care? > Because it go

Re: Finding size of Variable

2014-02-08 Thread David Hutto
On Sat, Feb 8, 2014 at 8:17 AM, Mark Lawrence wrote: > On 08/02/2014 02:48, Steven D'Aprano wrote: > >> On Thu, 06 Feb 2014 05:51:54 -0800, wxjmfauth wrote: >> >> Sorry, I'm only pointing you may lose memory when working with short >>> strings as it was explained. I really, very really, do not se

Re: Finding size of Variable

2014-02-08 Thread Mark Lawrence
On 08/02/2014 02:48, Steven D'Aprano wrote: On Thu, 06 Feb 2014 05:51:54 -0800, wxjmfauth wrote: Sorry, I'm only pointing you may lose memory when working with short strings as it was explained. I really, very really, do not see what is absurd or obsure in: sys.getsizeof('abc' + 'EURO') 46

Re: Finding size of Variable

2014-02-07 Thread Ethan Furman
On 02/07/2014 06:48 PM, Steven D'Aprano wrote: That is not a trade-off that the core developers have chosen to make, and I agree with them. Even though you haven't broken all the build-bots yet, you can still stop saying "them". ;) -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/pyth

Re: Finding size of Variable

2014-02-07 Thread Steven D'Aprano
On Thu, 06 Feb 2014 05:51:54 -0800, wxjmfauth wrote: > Sorry, I'm only pointing you may lose memory when working with short > strings as it was explained. I really, very really, do not see what is > absurd or obsure in: > sys.getsizeof('abc' + 'EURO') > 46 sys.getsizeof(('abc' + 'EURO')

Re: Finding size of Variable

2014-02-06 Thread wxjmfauth
Some mysterious problem with the "euro". Let's take a real "French" char. >>> sys.getsizeof('abc' + 'œ') 46 >>> sys.getsizeof(('abc' + 'œ').encode('utf-32')) 37 or a "German" char, ẞ >>> sys.getsizeof('abc' + '\N{LATIN CAPITAL LETTER SHARP S}') 46 >>> sys.getsizeof(('abc' + '\N{LATIN CAPITAL

Re: Finding size of Variable

2014-02-06 Thread wxjmfauth
Le jeudi 6 février 2014 12:10:08 UTC+1, Ned Batchelder a écrit : > On 2/6/14 5:15 AM, wxjmfa...@gmail.com wrote: > > > > > > sum([sys.getsizeof(c) for c in ['a', 'a EURO', 'aa EURO']*3]) > > > 336 > > sum([sys.getsizeof(c) for c in ['aa EURO aa EURO']*3]) > > > 150 > > s

Re: Finding size of Variable

2014-02-06 Thread Ned Batchelder
On 2/6/14 5:15 AM, wxjmfa...@gmail.com wrote: sum([sys.getsizeof(c) for c in ['a', 'a EURO', 'aa EURO']*3]) 336 sum([sys.getsizeof(c) for c in ['aa EURO aa EURO']*3]) 150 sum([sys.getsizeof(c.encode('utf-32')) for c in ['a', 'a EURO', 'aa EURO']*3]) 261 sum([sys.getsizeof(c.encode('utf-32

Re: Finding size of Variable

2014-02-06 Thread wxjmfauth
Le mercredi 5 février 2014 12:44:47 UTC+1, Chris Angelico a écrit : > On Wed, Feb 5, 2014 at 10:00 PM, Steven D'Aprano > > wrote: > > >> where stopWords.txt is a file of size 4KB > > > > > > My guess is that if you split a 4K file into words, then put the words > > > into a list, you'll proba

Re: Finding size of Variable

2014-02-05 Thread Mark Lawrence
On 05/02/2014 14:33, Ayushi Dalmia wrote: Please stop sending double line spaced messages, just follow the instructions here https://wiki.python.org/moin/GoogleGroupsPython to prevent this happening, thanks. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can

Re: Finding size of Variable

2014-02-05 Thread Ayushi Dalmia
On Wednesday, February 5, 2014 7:13:34 PM UTC+5:30, Dave Angel wrote: > Ayushi Dalmia Wrote in message: > > > On Wednesday, February 5, 2014 12:59:46 AM UTC+5:30, Tim Chase wrote: > > >> On 2014-02-04 14:21, Dave Angel wrote: > > >> > > >> > To get the "total" size of a list of strings, try

Re: Finding size of Variable

2014-02-05 Thread Dave Angel
Ayushi Dalmia Wrote in message: > On Wednesday, February 5, 2014 12:59:46 AM UTC+5:30, Tim Chase wrote: >> On 2014-02-04 14:21, Dave Angel wrote: >> >> > To get the "total" size of a list of strings, try (untested): >> >> > >> >> > a = sys.getsizeof (mylist ) >> >> > for item in mylist: >>

Re: Finding size of Variable

2014-02-05 Thread Chris Angelico
On Wed, Feb 5, 2014 at 10:00 PM, Steven D'Aprano wrote: >> where stopWords.txt is a file of size 4KB > > My guess is that if you split a 4K file into words, then put the words > into a list, you'll probably end up with 6-8K in memory. I'd guess rather more; Python strings have a fair bit of fixed

Re: Finding size of Variable

2014-02-05 Thread Steven D'Aprano
On Tue, 04 Feb 2014 21:35:05 -0800, Ayushi Dalmia wrote: > On Wednesday, February 5, 2014 12:59:46 AM UTC+5:30, Tim Chase wrote: >> On 2014-02-04 14:21, Dave Angel wrote: >> >> > To get the "total" size of a list of strings, try (untested): >> >> > >> > a = sys.getsizeof (mylist ) >> > for ite

Re: Finding size of Variable

2014-02-05 Thread Peter Otten
Ayushi Dalmia wrote: > On Wednesday, February 5, 2014 12:51:31 AM UTC+5:30, Dave Angel wrote: >> Ayushi Dalmia Wrote in message: >> >> >> >> > >> >> > Where am I going wrong? What are the alternatives I can try? >> >> >> >> You've rejected all the alternatives so far without showing your

Re: Finding size of Variable

2014-02-04 Thread Ayushi Dalmia
On Wednesday, February 5, 2014 11:15:09 AM UTC+5:30, Rustom Mody wrote: > On Wednesday, February 5, 2014 11:05:05 AM UTC+5:30, Ayushi Dalmia wrote: > > > This also doesn't gives the true size. I did the following: > > > > > import sys > > > data=[] > > > f=open('stopWords.txt','r') > > > >

Re: Finding size of Variable

2014-02-04 Thread Rustom Mody
On Wednesday, February 5, 2014 11:05:05 AM UTC+5:30, Ayushi Dalmia wrote: > This also doesn't gives the true size. I did the following: > import sys > data=[] > f=open('stopWords.txt','r') > for line in f: > line=line.split() > data.extend(line) > print sys.getsizeof(data) > where stopW

Re: Finding size of Variable

2014-02-04 Thread Ayushi Dalmia
On Wednesday, February 5, 2014 12:59:46 AM UTC+5:30, Tim Chase wrote: > On 2014-02-04 14:21, Dave Angel wrote: > > > To get the "total" size of a list of strings, try (untested): > > > > > > a = sys.getsizeof (mylist ) > > > for item in mylist: > > > a += sys.getsizeof (item) > > > >

Re: Finding size of Variable

2014-02-04 Thread Ayushi Dalmia
On Wednesday, February 5, 2014 12:51:31 AM UTC+5:30, Dave Angel wrote: > Ayushi Dalmia Wrote in message: > > > > > > > > Where am I going wrong? What are the alternatives I can try? > > > > You've rejected all the alternatives so far without showing your > > code, or even properly specif

Re: Finding size of Variable

2014-02-04 Thread Ayushi Dalmia
On Tuesday, February 4, 2014 7:36:48 PM UTC+5:30, Dennis Lee Bieber wrote: > On Tue, 4 Feb 2014 05:19:48 -0800 (PST), Ayushi Dalmia > > declaimed the following: > > > > > > >I need to chunk out the outputs otherwise it will give Memory Error. I need > >to do some postprocessing on the data

Re: Finding size of Variable

2014-02-04 Thread Tim Golden
On 04/02/2014 19:21, Dave Angel wrote: Ayushi Dalmia Wrote in message: Where am I going wrong? What are the alternatives I can try? You've rejected all the alternatives so far without showing your code, or even properly specifying your problem. To get the "total" size of a list of stri

Re: Finding size of Variable

2014-02-04 Thread Tim Chase
On 2014-02-04 14:21, Dave Angel wrote: > To get the "total" size of a list of strings, try (untested): > > a = sys.getsizeof (mylist ) > for item in mylist: > a += sys.getsizeof (item) I always find this sort of accumulation weird (well, at least in Python; it's the *only* way in many other

Re: Finding size of Variable

2014-02-04 Thread Ayushi Dalmia
On Tuesday, February 4, 2014 6:23:19 PM UTC+5:30, Asaf Las wrote: > On Tuesday, February 4, 2014 2:43:21 PM UTC+2, Ayushi Dalmia wrote: > > > > > > As I said, I need to merge large files and I cannot afford more I/O > > > operations. So in order to minimise the I/O operation I am writing in >

Re: Finding size of Variable

2014-02-04 Thread Ayushi Dalmia
On Tuesday, February 4, 2014 6:39:00 PM UTC+5:30, Dave Angel wrote: > Ayushi Dalmia Wrote in message: > > > > >> getsizeof() gives you the size of the list only; to complete the picture > >> you > > >> > > >> have to add the sizes of the lines. > > >> > > >> > > >> > > >> However,

Re: Finding size of Variable

2014-02-04 Thread Dave Angel
Ayushi Dalmia Wrote in message: >> getsizeof() gives you the size of the list only; to complete the picture you >> >> have to add the sizes of the lines. >> >> >> >> However, why do you want to keep track of the actual memory used by >> >> variables in your script? You should instead con

Re: Finding size of Variable

2014-02-04 Thread Asaf Las
On Tuesday, February 4, 2014 2:43:21 PM UTC+2, Ayushi Dalmia wrote: > > As I said, I need to merge large files and I cannot afford more I/O > operations. So in order to minimise the I/O operation I am writing in > chunks. Also, I need to use the merged files as indexes later which > should be l

Re: Finding size of Variable

2014-02-04 Thread Ayushi Dalmia
On Tuesday, February 4, 2014 5:10:25 PM UTC+5:30, Peter Otten wrote: > Ayushi Dalmia wrote: > > > > > I have 10 files and I need to merge them (using K way merging). The size > > > of each file is around 200 MB. Now suppose I am keeping the merged data in > > > a variable named mergedData, I h

Re: Finding size of Variable

2014-02-04 Thread Peter Otten
Ayushi Dalmia wrote: > I have 10 files and I need to merge them (using K way merging). The size > of each file is around 200 MB. Now suppose I am keeping the merged data in > a variable named mergedData, I had thought of checking the size of > mergedData using sys.getsizeof() but it somehow doesn'