Re: [Python-Dev] Optimization

[Stephen J. Turnbull]

Nick Coghlan writes:

 > I suspect "list" may have been the intended comparison there.
 > "array.array" is another appropriate comparison.  Having bytearray
 > operations differ in algorithmic complexity from those two types
 > would be very strange and surprising

I don't see why.  If you really wanted bytearrays to be similar to
list or array.array in that way, wouldn't you derive them from one of
those types[1] rather than provide a primitive?  Having a separate
primitive makes it easier to implement optimized performance
characteristics.  Isn't that why we have several different kinds of
containers instead of deriving everything from list?

For me, the point about string "+=" being efficient (sometimes) isn't
that it is surprising compared to similar types, it's that it is
surprising for any immutable sequence type.

Footnotes: 
[1]  Or all of them from an ABC.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Wildcard mask and ANY IP

[Amer]

Hello,

I would like to thank you for your great effort
I have this question:

I want to do the following IP address style in python

ANY.45.10.ANY
ANY.x.y.z

I tried something like:
10.45.10.1 0.255.255.0, but it is not accepted I got some errors

Any help is appreciated
Best regards,
Amer
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Reduce memory footprint of Python

[Victor Stinner]

Hi,

Slowly, I'm trying to see if it would be possible to reduce the memory
footprint of Python using the tracemalloc module.

First, I noticed that linecache can allocate more than 2 MB. What do
you think of adding a registry of "clear cache" functions? For
exemple, re.purge() and linecache.clearcache(). gc.collect() clears
free lists. I don't know if gc.collect() should be related to this new
registy (clear all caches) or not.

The dictionary of interned Unicode strings can be large: up to 1.5 MB
(with +30,000 strings). Just the dictionary, excluding size of
strings. Is the size normal or not? Using tracemalloc, this dictionary
is usually to largest memory block.

unittest doesn't look to release memory (the TestCase class) after the
execution of a test.

test_import.test_module_with_large_stack() creates a large Python
module and import it, but it does not unload it.

Should I open a separated issue for each idea to track them in the bug
tracker, or a global issue?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[Benjamin Peterson]

2013/10/6 Victor Stinner :
> Hi,
>
> Slowly, I'm trying to see if it would be possible to reduce the memory
> footprint of Python using the tracemalloc module.
>
> First, I noticed that linecache can allocate more than 2 MB. What do
> you think of adding a registry of "clear cache" functions? For
> exemple, re.purge() and linecache.clearcache(). gc.collect() clears
> free lists. I don't know if gc.collect() should be related to this new
> registy (clear all caches) or not.

What is the usecase for minimizing the memory usage of Python in the
middle of a program?

>
> The dictionary of interned Unicode strings can be large: up to 1.5 MB
> (with +30,000 strings). Just the dictionary, excluding size of
> strings. Is the size normal or not? Using tracemalloc, this dictionary
> is usually to largest memory block.

1.5 MB is sort of a pittance.

>
> unittest doesn't look to release memory (the TestCase class) after the
> execution of a test.

Is it important to optimize unittests for memory usage?


-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[Antoine Pitrou]

Hi,

On Sun, 6 Oct 2013 17:32:37 +0200
Victor Stinner  wrote:
> 
> Slowly, I'm trying to see if it would be possible to reduce the memory
> footprint of Python using the tracemalloc module.
> 
> First, I noticed that linecache can allocate more than 2 MB. What do
> you think of adding a registry of "clear cache" functions? For
> exemple, re.purge() and linecache.clearcache(). gc.collect() clears
> free lists. I don't know if gc.collect() should be related to this new
> registy (clear all caches) or not.

Rather than "clear" the cache, perhaps limit its size?
AFAICT, linecache is used in the warnings and traceback modules.

> The dictionary of interned Unicode strings can be large: up to 1.5 MB
> (with +30,000 strings). Just the dictionary, excluding size of
> strings. Is the size normal or not? Using tracemalloc, this dictionary
> is usually to largest memory block.

You don't tell us what the total memory occupation is. A 1.5MB overhead
is not the same on a 10MB total or on a 50MB total.
(also, how many modules were loaded? large third-party libraries?)

The interned strings dictionary could perhaps be replaced with a set?

> unittest doesn't look to release memory (the TestCase class) after the
> execution of a test.
> 
> test_import.test_module_with_large_stack() creates a large Python
> module and import it, but it does not unload it.

That's not really relevant to memory use on production systems, though.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[Georg Brandl]

Am 06.10.2013 17:32, schrieb Victor Stinner:
> Hi,
> 
> Slowly, I'm trying to see if it would be possible to reduce the memory
> footprint of Python using the tracemalloc module.
> 
> First, I noticed that linecache can allocate more than 2 MB. What do
> you think of adding a registry of "clear cache" functions? For
> exemple, re.purge() and linecache.clearcache(). gc.collect() clears
> free lists. I don't know if gc.collect() should be related to this new
> registy (clear all caches) or not.
> 
> The dictionary of interned Unicode strings can be large: up to 1.5 MB
> (with +30,000 strings). Just the dictionary, excluding size of
> strings. Is the size normal or not? Using tracemalloc, this dictionary
> is usually to largest memory block.
> 
> unittest doesn't look to release memory (the TestCase class) after the
> execution of a test.
> 
> test_import.test_module_with_large_stack() creates a large Python
> module and import it, but it does not unload it.

That does not seem very important, except if people execute test_import
on every interpreter startup :)

> Should I open a separated issue for each idea to track them in the bug
> tracker, or a global issue?

Since the points you mention here are so diverse, it would be good to have
separate issues.

cheers,
Georg

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[Victor Stinner]

2013/10/6 Benjamin Peterson :
> 2013/10/6 Victor Stinner :
>> Hi,
>>
>> Slowly, I'm trying to see if it would be possible to reduce the memory
>> footprint of Python using the tracemalloc module.
>>
>> First, I noticed that linecache can allocate more than 2 MB. What do
>> you think of adding a registry of "clear cache" functions? For
>> exemple, re.purge() and linecache.clearcache(). gc.collect() clears
>> free lists. I don't know if gc.collect() should be related to this new
>> registy (clear all caches) or not.
>
> What is the usecase for minimizing the memory usage of Python in the
> middle of a program?

If you know that your application uses a lot of memory, it is
interesting to sometimes (when the application is idle) try to release
some bytes to not use all the system memory. On embedded devices,
memory is expensive and very limited. Each byte is important :-)

>> unittest doesn't look to release memory (the TestCase class) after the
>> execution of a test.
>
> Is it important to optimize unittests for memory usage?

I was surprised that running the whole test suite use more and more
memory. I expected something more stable with some short peaks.

Reducing the memory footprint is useful for performances (of test
suite) and be able to run the test suite on system with low memory
(embedded devices).

Ezio pointed me the following issues:

http://bugs.python.org/issue17534
http://bugs.python.org/issue11798
http://bugs.python.org/issue9815

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[Benjamin Peterson]

2013/10/6 Victor Stinner :
> 2013/10/6 Benjamin Peterson :
>> 2013/10/6 Victor Stinner :
>>> Hi,
>>>
>>> Slowly, I'm trying to see if it would be possible to reduce the memory
>>> footprint of Python using the tracemalloc module.
>>>
>>> First, I noticed that linecache can allocate more than 2 MB. What do
>>> you think of adding a registry of "clear cache" functions? For
>>> exemple, re.purge() and linecache.clearcache(). gc.collect() clears
>>> free lists. I don't know if gc.collect() should be related to this new
>>> registy (clear all caches) or not.
>>
>> What is the usecase for minimizing the memory usage of Python in the
>> middle of a program?
>
> If you know that your application uses a lot of memory, it is
> interesting to sometimes (when the application is idle) try to release
> some bytes to not use all the system memory. On embedded devices,
> memory is expensive and very limited. Each byte is important :-)

How many embedded systems are running Python?


-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Wildcard mask and ANY IP

[Stephen J. Turnbull]

Amer writes:
 > Hello,
 > 
 > I would like to thank you for your great effort
 > I have this question:
 > 
 > I want to do the following IP address style in python
 > 
 > ANY.45.10.ANY
 > ANY.x.y.z

This list is for developing the Python language itself (including the
CPython interpreter and the standard library of modules).  This kind
of question is better addressed to python-l...@python.org (also
accessible from netnews as the comp.lang.python group).

Before you post again to an appropriate channel, your question cannot
be given a useful answer as it stands.  You need to include the code
you are using to parse the addresses, and perhaps some idea of what
kind of application you are writing.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[Stefan Behnel]

Benjamin Peterson, 06.10.2013 17:41:
> 2013/10/6 Victor Stinner:
>> unittest doesn't look to release memory (the TestCase class) after the
>> execution of a test.
> 
> Is it important to optimize unittests for memory usage?

Maybe "optimise" isn't the best word, but test suites can sometimes be huge
and I can't see a reason to keep an old TestClass instance around after all
its tests have run. So why not free it? If the tests do any intermediate
caching, it's likely to happen at that level (or at the module level, but
that's the user's decision).

Sure, if memory matters, there's tearDownClass() for manual cleanup, but
why require users to care when we otherwise provide automatic memory
management everywhere else? It certainly improves the default setup.

Stefan


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Optimization

[Xavier Morel]

On 2013-10-06, at 12:37 , Stephen J. Turnbull wrote:
> 
> For me, the point about string "+=" being efficient (sometimes) isn't
> that it is surprising compared to similar types, it's that it is
> surprising for any immutable sequence type.

It's clearly nitpicking, but ropes are immutable sequences with O(1)
concatenation. Clojure's vectors also have a more-or-less-O(1) append
(technically it's O(log32 n)) and one could implement a Python
version of them.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[Victor Stinner]

Benjamin wrote:
> Is it important to optimize unittests for memory usage?

2013/10/6 Georg Brandl :
> That does not seem very important, except if people execute test_import
> on every interpreter startup :)

Oh, I just realized that I forgot to explain why I'm starting with
unit tests. I ran the Python test suite using -X tracemalloc (tracing
all memory allocations) because the test suite is a nice tool to
stress a new module (tracemalloc). The test suite creates many
threads, subprocesses, send random signals, etc.

That's why I started to monitor the test suite. I know that it's not
the most critical part of Python :-)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[martin]

Quoting Benjamin Peterson :


If you know that your application uses a lot of memory, it is
interesting to sometimes (when the application is idle) try to release
some bytes to not use all the system memory. On embedded devices,
memory is expensive and very limited. Each byte is important :-)


How many embedded systems are running Python?


And of those, how many use the linecache module?

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[Antoine Pitrou]

On Sun, 06 Oct 2013 22:27:52 +0200
mar...@v.loewis.de wrote:
> 
> Quoting Benjamin Peterson :
> 
> >> If you know that your application uses a lot of memory, it is
> >> interesting to sometimes (when the application is idle) try to release
> >> some bytes to not use all the system memory. On embedded devices,
> >> memory is expensive and very limited. Each byte is important :-)
> >
> > How many embedded systems are running Python?
> 
> And of those, how many use the linecache module?

The linecache module is used implicitly by the traceback and warnings
module, so perhaps quite a bit more than one would imagine :-)

I think limiting the linecache cache size would be good enough.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[martin]

Quoting Victor Stinner :


Slowly, I'm trying to see if it would be possible to reduce the memory
footprint of Python using the tracemalloc module.

[...]

Should I open a separated issue for each idea to track them in the bug
tracker, or a global issue?


There is a third alternative which I would recommend: not open tracker
issues at all - unless you can also offer a patch. The things you find
are not bugs per se, not even "issues". It is fine and applaudable that
you look into this, but other people may have other priorities (like
reimplementing the hash function of string objects).

So if you remember that there is a potential for optimization, that
may be enough for the moment. Or share it on python-dev (as you do
below); people may be intrigued to look into this further, or ignore
it. It's easy to ignore a posting to python-dev, but more difficult to
ignore an issue on the tracker (*something* should be done about it,
e.g. close with no action).


First, I noticed that linecache can allocate more than 2 MB. What do
you think of adding a registry of "clear cache" functions? For
exemple, re.purge() and linecache.clearcache(). gc.collect() clears
free lists. I don't know if gc.collect() should be related to this new
registy (clear all caches) or not.


I'm -1 on this idea. There are some "canonical" events that could trigger
clearance of caches, namely
- out-of-memory situations
- OS signals indicating memory pressure
While these sound interesting in theory, they fail in practice. For
example, they are very difficult to test.


The dictionary of interned Unicode strings can be large: up to 1.5 MB
(with +30,000 strings). Just the dictionary, excluding size of
strings. Is the size normal or not? Using tracemalloc, this dictionary
is usually to largest memory block.


I'd check the contents of the dictionary. How many strings are in there;
how many of these are identifiers; how many have more than one outside
reference; how many are immortal?

If there is a lot of strings that are not identifiers, some code possibly
abuses interning, and should use its own dictionary instead. For the
refcount-1 mortal identifiers, I'd trace back where they are stored,
and check if many of them originate from the same module.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[martin]

Quoting Antoine Pitrou :


The linecache module is used implicitly by the traceback and warnings
module, so perhaps quite a bit more than one would imagine :-)

I think limiting the linecache cache size would be good enough.


So what specific limit would you suggest?

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[Antoine Pitrou]

On Sun, 06 Oct 2013 22:47:27 +0200
mar...@v.loewis.de wrote:
> 
> Quoting Antoine Pitrou :
> 
> > The linecache module is used implicitly by the traceback and warnings
> > module, so perhaps quite a bit more than one would imagine :-)
> >
> > I think limiting the linecache cache size would be good enough.
> 
> So what specific limit would you suggest?

Something like 50 files.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[Victor Stinner]

2013/10/6  :
> Quoting Antoine Pitrou :
>
>> The linecache module is used implicitly by the traceback and warnings
>> module, so perhaps quite a bit more than one would imagine :-)
>>
>> I think limiting the linecache cache size would be good enough.
>
> So what specific limit would you suggest?

Before using tracemalloc, I didn't even know that lineache has a
cache. And Antoine wrote, linecache is used indirectly by many
modules. Even if you get a single line, the module does all the whole
file content. I was surprised to see that the cache size can be up to
2 MB.

(To answer to your question, yes, our application on embedded device
does format exceptions and tracebacks, so use linecache.)

The re cache is limited. I remember that I saw it once bigger than 700
KB, it was to build a huge range of Unicode characters... but I'm now
unable to reproduce the case.

I'm quite sure that the cache has an impact on performances. For
example, if you format a traceback where most frames come from the
same file, you don't want to decode the file from Unicode for each
line. I don't want to remove the cache, and I'm unable to configure
its size.

I suggested a function to clear "all" caches, because *if* you care of
memory, you would like to flush all caches (even if you don't know
that linecache or re have caches), not anytime, but when your
application is idle. I propose a registry because IMO the module
should register itself to the registry, the registry should not have
to know all modules. (It should not the module to clear its cache ;-))

I worked on a project spending weeks to try to reduce the memory
footprint by 1 MB. And it was hard :-) Biggest objects were in the
application but it would help to be able to free Python internal
caches.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] intobject.c free_list and int_dealloc()

[Laurent Luce]

Hello,

I am trying to understand how the free list linked-list evolves when an int 
object is deallocated.

At the beginning, free_list is pointing to the last int object (n-1) in the 
block.  We initialize two int objects, free_list now points to the int object: 
n-3.

free_list -> n-3
n-1 -> n-2
n-2 -> n-3
n-3 -> n-4...


We delete the first int object initialized:


n-1 -> n-3
free_list -> n-1

It seems to me that two elements are now pointing to the n-3 element: n-1 and 
n-2.  I am pretty sure I am missing something here.  Can someone let me know?


Thanks,


Laurent___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[Olemis Lang]

On 10/6/13, Benjamin Peterson  wrote:
> 2013/10/6 Victor Stinner :
>> Hi,
>>

:)

[...]
>>
>> unittest doesn't look to release memory (the TestCase class) after the
>> execution of a test.
>
> Is it important to optimize unittests for memory usage?
>

AFAICT , test results will stored the outcomes of individual test
cases , which is O(n) under normal circumstances, but I guess this is
not what Victor was talking about (isn't it ?).

However , I've been thinking since some time ago that if the only
outcome of running the test suite is to dump the output to stdout or
any other file-like object then test result lists might be of little
value most of the time . Maybe an efficient runner + results
implementation focused on streaming output with no caching could help
with efficient memory allocations .

Regarding the memory allocated in setUp* method(s) then IMO it should
be torn down if it will not be used anymore. Such optimizations should
be better performed in TCs tearDown* methods themselves rather than
e.g. automatically deallocating memory associated to TC instances .
Sometimes different tools use such data for certain analysis .


BTW , I've detected a few instances where this proves to be useful (to
me) ; especially (concurrently) running (matrix jobs of) relatively
long test suites on hosted (Jenkins) instances , where quotas apply .


-- 
Regards,

Olemis - @olemislc

Apache™ Bloodhound contributor
http://issues.apache.org/bloodhound
http://blood-hound.net

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Popularidad de Python, septiembre 2013 -
http://goo.gl/fb/tr0XB
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Fwd: Reduce memory footprint of Python

[Olemis Lang]

forwarding to the list , sorry ...

-- Forwarded message --
From: Olemis Lang 
Date: Sun, 6 Oct 2013 17:09:38 -0500
Subject: Re: [Python-Dev] Reduce memory footprint of Python
To: Benjamin Peterson 

On 10/6/13, Benjamin Peterson  wrote:
> 2013/10/6 Victor Stinner :
>> 2013/10/6 Benjamin Peterson :
>>> 2013/10/6 Victor Stinner :
 Hi,

 Slowly, I'm trying to see if it would be possible to reduce the memory
 footprint of Python using the tracemalloc module.

 First, I noticed that linecache can allocate more than 2 MB. What do
 you think of adding a registry of "clear cache" functions? For
 exemple, re.purge() and linecache.clearcache(). gc.collect() clears
 free lists. I don't know if gc.collect() should be related to this new
 registy (clear all caches) or not.
>>>
>>> What is the usecase for minimizing the memory usage of Python in the
>>> middle of a program?
>>
>> If you know that your application uses a lot of memory, it is
>> interesting to sometimes (when the application is idle) try to release
>> some bytes to not use all the system memory. On embedded devices,
>> memory is expensive and very limited. Each byte is important :-)
>
> How many embedded systems are running Python?
>


I know of a few of them, and it seems they will be more popular with
the growing interest for devices like Raspberry Pi and technologies
like 3D printing e.g. http://raspberry-python.blogspot.com/


-- 
Regards,

Olemis - @olemislc

Apache™ Bloodhound contributor
http://issues.apache.org/bloodhound
http://blood-hound.net

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Popularidad de Python, septiembre 2013 -
http://goo.gl/fb/tr0XB
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython (3.3): PythonCAD is now on PyQt, use Wing as a prominent PyGtk example.

[Michael Foord]

On 6 October 2013 11:45, georg.brandl  wrote:

> http://hg.python.org/cpython/rev/942b9420e7e9
> changeset:   86065:942b9420e7e9
> branch:  3.3
> parent:  86062:6ced4fb4f711
> user:Georg Brandl 
> date:Sun Oct 06 12:46:13 2013 +0200
> summary:
>   PythonCAD is now on PyQt, use Wing as a prominent PyGtk example.
>
>

Wing is only a good example of PyGtk until Wing 5 is out of beta. They too
have switched to PyQt...

Michael


> Found by Helge Stenström on docs@.
>
> files:
>   Doc/library/othergui.rst |  2 +-
>   1 files changed, 1 insertions(+), 1 deletions(-)
>
>
> diff --git a/Doc/library/othergui.rst b/Doc/library/othergui.rst
> --- a/Doc/library/othergui.rst
> +++ b/Doc/library/othergui.rst
> @@ -20,7 +20,7 @@
>of the library, GTK+ 2.  It provides an object oriented interface
> that
>is slightly higher level than the C one.  There are also bindings to
>`GNOME `_.  One well known PyGTK application
> is
> -  `PythonCAD `_. An online `tutorial
> +  `WingIDE `_. An online `tutorial
>`_ is available.
>
> `PyQt `_
>
> --
> Repository URL: http://hg.python.org/cpython
>
> ___
> Python-checkins mailing list
> python-check...@python.org
> https://mail.python.org/mailman/listinfo/python-checkins
>
>


-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 454: Add a new tracemalloc module (final version)

[Victor Stinner]

2013/10/3 Victor Stinner :
> I worked on the implementation of the tracemalloc module and its PEP
> 454. I consider that this third version of the PEP is ready for a
> final review.
>
> HTML version of the PEP 454:
> http://www.python.org/dev/peps/pep-0454/

About the implementation.

I'm working in this repository:
http://hg.python.org/features/tracemalloc

I updated the patch for Python default on the issue:
http://bugs.python.org/issue18874

I backported my new version of the module on Python 2.7 and 3.3:
https://github.com/haypo/pytracemalloc

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Make str/bytes hash algorithm pluggable?

[Nick Coghlan]

On 4 Oct 2013 07:17, "Guido van Rossum"  wrote:
>
> On Thu, Oct 3, 2013 at 2:13 PM, Nick Coghlan  wrote:
>>
>> On 4 Oct 2013 06:08, "Victor Stinner"  wrote:
>> >
>> > 2013/10/3 Christian Heimes :
>> > > A hash algorithm can be added and one avaible hash
>> > > algorithm can be set before Py_Initialize() is called for the first
>> > > time.
>> >
>> > "Py_Initialize" is not the good guard. Try for example "python3 -X
>> > faulthandler": PyObject_Hash() is called before Py_Initialize() to add
>> > "faulthandler" key into sys._xoptions dictionary.
>> >
>> > Today many Python internal functions are used before Python is
>> > initialized... See the PEP 432 which proposes to improve the
>> > situation:
>> > http://www.python.org/dev/peps/pep-0432/
>>
>> That problem exists because our main function doesn't follow the C API
usage rules, though. We require other embedding applications to be better
behaved than that if they want support :)
>>
>> That said, while I'm mostly in favour of the PEP, I think setting the
algorithm should be a private API for 3.4.
>>
>> I do agree that since the platform support for SipHash is slightly
narrower,  we need to keep the existing hash algorithm around, make it
relatively easy to enable and ensure we continue to test it on the build
bots.
>>
>> I believe that last requirement for buildbot testing is the one that
should drive the design of the private configuration API.
>
> I'll defer to Nick for approval of this PEP.

Sure, I'm happy to be BDFL delegate for this one. Christian, go ahead and
add the appropriate header to the PEP.

Cheers,
Nick.

>
> --
> --Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

[Nick Coghlan]

On 7 Oct 2013 07:28, "Victor Stinner"  wrote:
>
> 2013/10/6  :
> > Quoting Antoine Pitrou :
> >
> >> The linecache module is used implicitly by the traceback and warnings
> >> module, so perhaps quite a bit more than one would imagine :-)
> >>
> >> I think limiting the linecache cache size would be good enough.
> >
> > So what specific limit would you suggest?
>
> Before using tracemalloc, I didn't even know that lineache has a
> cache. And Antoine wrote, linecache is used indirectly by many
> modules. Even if you get a single line, the module does all the whole
> file content. I was surprised to see that the cache size can be up to
> 2 MB.
>
> (To answer to your question, yes, our application on embedded device
> does format exceptions and tracebacks, so use linecache.)
>
> The re cache is limited. I remember that I saw it once bigger than 700
> KB, it was to build a huge range of Unicode characters... but I'm now
> unable to reproduce the case.
>
> I'm quite sure that the cache has an impact on performances. For
> example, if you format a traceback where most frames come from the
> same file, you don't want to decode the file from Unicode for each
> line. I don't want to remove the cache, and I'm unable to configure
> its size.
>
> I suggested a function to clear "all" caches, because *if* you care of
> memory, you would like to flush all caches (even if you don't know
> that linecache or re have caches), not anytime, but when your
> application is idle. I propose a registry because IMO the module
> should register itself to the registry, the registry should not have
> to know all modules. (It should not the module to clear its cache ;-))

We kind of have such a manual registry already: the reference leak hunting
in regrtest needs to know about a lot of these caches so it can clear them.

Something like a "gc.purge_caches" operation with a "gc.atpurge"
registration function sounds like a reasonable operation to me (especially
if it offered a way to get a list of which modules had registered cache
purging functions).

Like any import time side effects, we'd need to look carefully at the
impact it has on the lifecycle of the module globals. We'd also need to be
sure that any caches we registered really were pure caches.

Cheers,
Nick.

>
> I worked on a project spending weeks to try to reduce the memory
> footprint by 1 MB. And it was hard :-) Biggest objects were in the
> application but it would help to be able to free Python internal
> caches.
>
> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com