This is presumably the same memory leak as in
https://trac.sagemath.org/ticket/31340
On Thursday, March 18, 2021 at 7:21:14 PM UTC+1 m.derick...@gmail.com wrote:
> Hi Vincent,
>
> Thanks for testing and good that you realized that indeed looked fishy
> indeed.
>
> I think your extension of my code with multiple loops actually confirms
> that there is a problem even after warming up:
>
> the prints:
>
> memory usage 30k: 1771.8046875
> memory usage 40k: 1771.80859375
>
> mean that you are using 1.7 GB memory extra after the 3th and the 4th loop
> (notice you chance the baseline of memory usage every loop so the 1.7 GB is
> not the total memory usage of sage, it is the new memory usage relative to
> the previous loop). So that in the end sage was using around 885+3*1777 MB
> = 6.1 GB of memory in your session.
>
> Also, the largest object ever created are a range object of size 10000
> (which in python 3 is actually consuming almost no memory since under the
> hood it behaves very much like xrange from python 2) and an srange object
> of size 3000. Neither of these should occupy an amount of memory in the GB
> range.
>
> Also one thing that is maybe easy to miss, I doubled the size of the loop
> between the first loop and the second loop, which explains the 885 vs 1771.
>
>
> On Thursday, 18 March 2021 at 17:52:19 UTC+1 vdelecroix wrote:
>
>> Sorry, your example indeed looks fishy.
>>
>> Le 18/03/2021 à 17:36, Vincent Delecroix a écrit :
>> > Maarten, in your example your can not fairly compare the first
>> > and second run. For example R is not allocated at your first
>> > call of "mem = get_memory_usage()". Also Integers have a pool
>> > which is likely not being filled at startup. Ignoring the first
>> > run, I do not notice any difference.
>> >
>> > (both on system sage 9.2 and compiled sage 9.3.beta8 on archlinux)
>> >
>> > sage: M = 3000
>> > ....: import gc
>> > ....: _ = gc.collect()
>> > ....: mem = get_memory_usage()
>> > ....: for i in range(10000):
>> > ....: R = srange(M)
>> > ....: _ = gc.collect()
>> > ....: print("memory usage 10k:", get_memory_usage(mem))
>> > ....: mem = get_memory_usage()
>> > ....: for i in range(20000):
>> > ....: R = srange(M)
>> > ....: _ = gc.collect()
>> > ....: print("memory usage 20k:", get_memory_usage(mem))
>> > ....: mem = get_memory_usage()
>> > ....: for i in range(20000):
>> > ....: R = srange(M)
>> > ....: _ = gc.collect()
>> > ....: print("memory usage 30k:", get_memory_usage(mem))
>> > ....: mem = get_memory_usage()
>> > ....: for i in range(20000):
>> > ....: R = srange(M)
>> > ....: _ = gc.collect()
>> > ....: print("memory usage 40k:", get_memory_usage(mem))
>> >
>> > memory usage 10k: 885.4453125
>> > memory usage 20k: 1771.9375
>> > memory usage 30k: 1771.8046875
>> > memory usage 40k: 1771.80859375
>> >
>> > Le 18/03/2021 à 17:00, Maarten Derickx a écrit :
>> >> Hi Guys,
>> >>
>> >> Thanks for the replies. I think this is enough info to know that it
>> >> happens
>> >> on multiple systems and that it's not just the cocalc enhanced version
>> of
>> >> sage. I have created:
>> >> https://trac.sagemath.org/ticket/31511#ticket for this.
>> >>
>> >> In the meantime I found the problem is already with the srange
>> >> command. So
>> >> the integer multiplication seems to be a red herring.
>> >>
>> >> ~$ sage
>> >> ┌────────────────────────────────────────────────────────────────────┐
>> >> │ SageMath version 9.2, Release Date: 2020-10-24 │
>> >> │ Create a "Sage Worksheet" file for the notebook interface. │
>> >> │ Enhanced for CoCalc. │
>> >> │ Using Python 3.8.5. Type "help()" for help. │
>> >> └────────────────────────────────────────────────────────────────────┘
>> >> sage: M = 3000
>> >> ....: import gc
>> >> ....: gc.collect()
>> >> ....: mem = get_memory_usage()
>> >> ....: for i in range(10000):
>> >> ....: R = srange(M)
>> >> ....: gc.collect()
>> >> ....: print("memory usage 10k:", get_memory_usage(mem))
>> >> ....: mem = get_memory_usage()
>> >> ....: for i in range(20000):
>> >> ....: R = srange(M)
>> >> ....: gc.collect()
>> >> ....: print("memory usage 20k:", get_memory_usage(mem))
>> >> 434
>> >> 0
>> >> memory usage 10k: 884.4296875
>> >> 0
>> >> memory usage 20k: 1770.71875
>> >>
>> >> On Thursday, 18 March 2021 at 16:01:56 UTC+1 David Joyner wrote:
>> >>
>> >>> On Thu, Mar 18, 2021 at 6:56 AM Maarten Derickx <
>> m.derick...@gmail.com>
>> >>> wrote:
>> >>>
>> >>>> Hi All,
>> >>>>
>> >>>> tldr: the bottom of this post contains example code of which I would
>> >>>> like
>> >>>> the results on some other systems.
>> >>>>
>> >>>> I recently encountered a memory leak in the relatively innocently
>> >>>> looking
>> >>>> code:
>> >>>>
>> >>>> d = 27
>> >>>> M = 109
>> >>>> for i in range(10000):
>> >>>> for a in srange(M):
>> >>>> for r in srange(d):
>> >>>> tmp = r*a
>> >>>>
>> >>>> However it doesn't seem to be on all platforms that sage supports.
>> For
>> >>>> example on cocalc (ubuntu 20.04) one gets:
>> >>>>
>> >>>> ~$ sage
>> >>>>
>> ┌────────────────────────────────────────────────────────────────────┐
>> >>>> │ SageMath version 9.2, Release Date: 2020-10-24
>> │
>> >>>> │ Create a "Sage Worksheet" file for the notebook interface.
>> │
>> >>>> │ Enhanced for CoCalc.
>> │
>> >>>> │ Using Python 3.8.5. Type "help()" for help.
>> │
>> >>>>
>> └────────────────────────────────────────────────────────────────────┘
>> >>>> sage: import gc
>> >>>> ....: gc.collect()
>> >>>> ....: mem = get_memory_usage()
>> >>>> ....: d = 27
>> >>>> ....: M = 109
>> >>>> ....: for i in range(10000):
>> >>>> ....: for a in srange(M):
>> >>>> ....: for r in srange(1,d):
>> >>>> ....: tmp = r*a
>> >>>> ....: gc.collect()
>> >>>> ....: print("memory usage 10k:", get_memory_usage(mem))
>> >>>> ....: mem = get_memory_usage()
>> >>>> ....: for i in range(20000):
>> >>>> ....: for a in srange(M):
>> >>>> ....: for r in srange(1,d):
>> >>>> ....: tmp = r*a
>> >>>> ....: gc.collect()
>> >>>> ....: print("memory usage 20k:", get_memory_usage(mem))
>> >>>> ....:
>> >>>> 434
>> >>>> 0
>> >>>> memory usage 10k: 9.15234375
>> >>>> 0
>> >>>> memory usage 20k: 21.3984375
>> >>>>
>> >>>> showing clear indication of a memory leak. While on my own laptop
>> (OS X
>> >>>> 10.13.6) I get:
>> >>>>
>> >>>>
>> >>>>
>> >>>> ~ mderickx$ sage
>> >>>>
>> ┌────────────────────────────────────────────────────────────────────┐
>> >>>> │ SageMath version 9.2, Release Date: 2020-10-24 │
>> >>>> │ Using Python 3.8.5. Type "help()" for help. │
>> >>>>
>> └────────────────────────────────────────────────────────────────────┘
>> >>>> sage: import gc
>> >>>> ....: gc.collect()
>> >>>> ....: mem = get_memory_usage()
>> >>>> ....: d = 27
>> >>>> ....: M = 109
>> >>>> ....: for i in range(10000):
>> >>>> ....: for a in srange(M):
>> >>>> ....: for r in srange(1,d):
>> >>>> ....: tmp = r*a
>> >>>> ....: gc.collect()
>> >>>> ....: print("memory usage 10k:", get_memory_usage(mem))
>> >>>> ....: mem = get_memory_usage()
>> >>>> ....: for i in range(20000):
>> >>>> ....: for a in srange(M):
>> >>>> ....: for r in srange(1,d):
>> >>>> ....: tmp = r*a
>> >>>> ....: gc.collect()
>> >>>> ....: print("memory usage 20k:", get_memory_usage(mem))
>> >>>> ....:
>> >>>> 278
>> >>>> 0
>> >>>> memory usage 10k: 0.0
>> >>>> 0
>> >>>> memory usage 20k: 0.0
>> >>>>
>> >>>>
>> >>>> so here the memory leak does not occur. However I don't have any
>> other
>> >>>> systems to which I have access to. So I was wondering if people
>> could
>> >>>> execute the code below on some other systems and report back here to
>> >>>> see
>> >>>> where the leaking is actually occurring. Since I do not have access
>> >>>> to a
>> >>>> plain vanilla sage install on ubuntu, *I am especially interested if
>> on
>> >>>> ubuntu the non cocalc enhanced version of sage also produces a
>> >>>> memory leak
>> >>>> for the following code:*
>> >>>>
>> >>>> import gc
>> >>>> gc.collect()
>> >>>> mem = get_memory_usage()
>> >>>> d = 27
>> >>>> M = 109
>> >>>> for i in range(10000):
>> >>>> for a in srange(M):
>> >>>> for r in srange(1,d):
>> >>>> tmp = r*a
>> >>>> gc.collect()
>> >>>> print("memory usage 10k:", get_memory_usage(mem))
>> >>>> mem = get_memory_usage()
>> >>>> for i in range(20000):
>> >>>> for a in srange(M):
>> >>>> for r in srange(1,d):
>> >>>> tmp = r*a
>> >>>> gc.collect()
>> >>>> print("memory usage 20k:", get_memory_usage(mem))
>> >>>>
>> >>>>
>> >>> This is in sage 9.1.rc5 running on ubuntu 20.4 on a dell inspiron
>> >>> laptop.
>> >>> It's an old version of sage but I hope it helps:
>> >>>
>> >>>
>> >>> sage: gc.collect()
>> >>> ....: mem = get_memory_usage()
>> >>> ....: d = 27
>> >>> ....: M = 109
>> >>> ....: for i in range(10000):
>> >>> ....: for a in srange(M):
>> >>> ....: for r in srange(1,d):
>> >>> ....: tmp = r*a
>> >>> ....: gc.collect()
>> >>> ....: print("memory usage 10k:", get_memory_usage(mem))
>> >>> ....: mem = get_memory_usage()
>> >>> ....: for i in range(20000):
>> >>> ....: for a in srange(M):
>> >>> ....: for r in srange(1,d):
>> >>> ....: tmp = r*a
>> >>> ....: gc.collect()
>> >>> ....: print("memory usage 20k:", get_memory_usage(mem))
>> >>> ....:
>> >>> 176
>> >>> 0
>> >>> memory usage 10k: 10.0546875
>> >>> 0
>> >>> memory usage 20k: 21.3984375
>> >>>
>> >>>
>> >>>
>> >>>> --
>> >>>> You received this message because you are subscribed to the Google
>> >>>> Groups
>> >>>> "sage-devel" group.
>> >>>> To unsubscribe from this group and stop receiving emails from it,
>> >>>> send an
>> >>>> email to sage-devel+...@googlegroups.com.
>> >>>> To view this discussion on the web visit
>> >>>>
>> https://groups.google.com/d/msgid/sage-devel/5882460b-842c-49c4-b33b-fae1c7986db9n%40googlegroups.com
>>
>> >>>>
>> >>>> <
>> https://groups.google.com/d/msgid/sage-devel/5882460b-842c-49c4-b33b-fae1c7986db9n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>
>>
>> >>>>
>> >>>> .
>> >>>>
>> >>>
>> >>
>>
>
--
You received this message because you are subscribed to the Google Groups
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/sage-devel/e7f09639-ec31-4e78-afae-f83c36d10063n%40googlegroups.com.
