Normal dict behavior?
Hi,
Python 2.7.5 (default, Nov 20 2013, 14:20:58)
[GCC 4.7.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> {0.: None, 0:None}
{0.0: None}
The second item disappeared!
Why?
Is it normal?
--
https://mail.python.org/mailman/listinfo/python-list
os.spawn* on Windows
Hello list
I am using Python 2.4 to invoke other programs on Windows XP, using
syntax like this:
os.spawnv(P_WAIT, "C:\\my.exe", ("C:\\my.exe",))
where my.exe is a console-mode program.
When the above statement is executed, the my.exe program creates a
window, what used to be called a Dos prompt but is no doubt called
something else now, and steals the keyboard input focus. This proves to
be highly annoying (because the window pops up unexpectedly in front of
my other windows and often steals several characters of what I am
typing).
I would rather the 'Dos prompt' was created minimised and did not steal
the input focus. Is there any way to achieve this end by using a
different Python command to invoke my.exe? Or would I have to change
my.exe itself?
Cheers
A.
--
http://mail.python.org/mailman/listinfo/python-list
SocketServer: replace network by hard drive
Hello, I would like to create a python server for which the requests are passed by files on the hard drive instead of a network. I am currently looking at the SocketServer python module, hoping for an easy modification. Is it doable at all? If yes, how should it be done? Thanks, Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Ordered dictionaries compared
Dan Stromberg gmail.com> writes: > > What kind of ordered dictionaries? Sorted by key. Calling them "sorted dictionaries" avoids any confusions with Python's standard OrderedDict class: http://docs.python.org/3.3/library/collections.html#ordereddict-objects Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: usage of os.posix_fadvise
Hi,
Wolfgang Maier biologie.uni-freiburg.de> writes:
>
> Dear all,
> I was just experimenting for the first time with os.posix_fadvise(), which
> is new in Python3.3 . I'm reading from a really huge file (several GB) and I
> want to use the data only once, so I don't want OS-level page caching. I
> tried os.posix_fadvise with the os.POSIX_FADV_NOREUSE and with the
> os.POSIX_FADV_DONTNEED flags, but neither seemed to have any effect on the
> caching behaviour of Ubuntu (still uses all available memory to page cache
> my I/O).
> Specifically, I was trying this:
>
> import os
> fd = os.open('myfile', os.O_RDONLY)
> # wasn't sure about the len parameter in fadvise,
> # so thought I just use it on the first 4GB
> os.posix_fadvise(fd, 0, 40, os.POSIX_FADV_NOREUSE) # or DONTNEED
The Linux version of "man posix_fadvise" probably holds the answer:
"In kernels before 2.6.18, POSIX_FADV_NOREUSE had the same semantics
as POSIX_FADV_WILLNEED. This was probably a bug; since kernel
2.6.18, this flag is a no-op."
"POSIX_FADV_DONTNEED attempts to free cached pages associated with the
specified region. This is useful, for example, while streaming large
files. A program may periodically request the kernel to free cached
data that has already been used, so that more useful cached pages are
not discarded instead."
So, in summary:
- POSIX_FADV_NOREUSE doesn't do anything on (modern) Linux kernels
- POSIX_FADV_DONTNEED must be called *after* you are done with a range of
data, not before you read it (note that I haven't tested to confirm it :-))
Regards
Antoine.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
Roy Smith panix.com> writes: > > Every line which contains 'ENQ' also matches the full regex (61425 > lines match, out of 2.1 million total). I don't understand why the > first way is so much slower. One invokes a fast special-purpose substring searching routine (the str.__contains__ operator), the other a generic matching engine able to process complex patterns. It's hardly a surprise for the specialized routine to be faster. That's like saying "I don't understand why my CPU is slower than my GPU at calculating 3D structures". That said, there may be ways to improve the regex engine to deal with such special cases in a speedier way. But there will still be some overhead related to the fact that you are invoking a powerful generic engine rather than a lean and mean specialized routine. (to be fair, on CPython there's also the fact that operators are faster than method calls, so some overhead is added by that too) > Once the regex is compiled, you should have a state machine pattern > matcher. It should be O(n) in the length of the input to figure out > that it doesn't match as far as "ENQ". And that's exactly how long it > should take for "if 'ENQ' not in line" to run as well. You should read again on the O(...) notation. It's an asymptotic complexity, it tells you nothing about the exact function values at different data points. So you can have two O(n) routines, one of which always twice faster than the other. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: ssl handshake operation timed out on Python 3.3.2
Benedict Verheyen gmail.com> writes: > > Hi, > > for a project, I need to post data to some aspx pages. > The aspx pages are hosted by another company. > I develop on a virtual Debian Wheezy (Virtual box) running on Windows. > I couldn't get the code to run either on Windows nor Linux. > > On my testserver (also a Debian Linux Wheezy) however, the code works with > Python 2.7.3 but not with Python 3.3.2. Another difference is that the > testserver has a direct ip, so nothing in between the machine and the > internet. I checked the firewall, the traffic passes without a problem. > > The error I get is that the SSL handshake operation timed out. > I tried some code that specifies the ssl protocol but the results are the > same. This may be a IIS-specific problem. Take a look at http://stackoverflow.com/questions/16365483/iis-7-5-mercurial-setup-ignoring-maxallowedcontentlength http://bz.selenic.com/show_bug.cgi?id=3905 http://bugs.python.org/issue17948 Otherwise, dumping network traffic with Wireshark could give you some hints at to what is different between the SSL handshakes in the two setups. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Stack Overflow moderator “animuson”
Joshua Landau landau.ws> writes: > > > The same with Unicode. We hate French people, > > And for good damn reason too. They're ruining our language, á mon avis. We do! Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why on CentOS, python consumes too much memory ?
gmail.com> writes: > > Hi: > >Previously, we found that our python scripts consume too much memory. So I use python's resource module to > restrict RLIMIT_AS's soft limit and hard limit to 200M. > On my RHEL5.3(i386)+python2.6.2, it works OK. But on CentOS 6.2(x86_64)+python2.6.6, it reports memory > error(exceeding 200M). Take a look at http://www.selenic.com/smem/ for accurate measurement of actual memory consumption under Linux. Virtual memory size is generally useless for this purpose. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: MyOpenID.com no longer supported
Le Mon, 29 Jul 2013 00:55:53 -0700, Ethan Furman a écrit : > Excerpt from http://meta.stackoverflow.com/q/190442/176681: > > Janrain no longer actively supports MyOpenID, and announced on > Twitter that their users should proceed with caution. > > This decision was made by Janrain, [snip] > > I know the Python bug tracker allows MyOpenID logins; if that is your > only login method you should add another that doesn't depend on > MyOpenID. I don't understand. The tracker allows any (sufficently compliant) OpenID provider, not just "MyOpenID" or "MyCorporateOpenIDWithTrademarks". Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with psycopg2, bytea, and memoryview
Frank Millman chagford.com> writes: > > I have some binary data (a gzipped xml object) that I want to store in a > database. For PostgreSQL I use a column with datatype 'bytea', which is > their recommended way of storing binary strings. > > I use psycopg2 to access the database. It returns binary data in the form of > a python 'memoryview'. > [...] > > Using MS SQL Server and pyodbc, it returns a byte string, not a memoryview, > and it does compare equal with the original. > > I can hack my program to use tobytes(), but it would add complication, and > it would be database-specific. I would prefer a cleaner solution. Just cast the result to bytes (`bytes(row[1])`). It will work both with bytes and memoryview objcts. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with psycopg2, bytea, and memoryview
Frank Millman chagford.com> writes: > > Thanks for that, Antoine. It is an improvement over tobytes(), but i am > afraid it is still not ideal for my purposes. I would suggest asking the psycopg2 project why they made this choice, and if they would reconsider. Returning a memoryview doesn't make much sense IMHO. For example, the standard sqlite3 module returns bytes for BLOB columns, and str for TEXT columns: http://docs.python.org/3.4/library/sqlite3.html#introduction > Can anyone explain *why* the results do not compare equal? If I understood > the problem, I might be able to find a workaround. Well, under recent Python versions, they should compare equal: Python 3.2.3 (default, Oct 19 2012, 19:53:16) [GCC 4.7.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> memoryview(b"abc") == b"abc" True Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: append in IMAP4 from imaplib very slow
Simon Pirschel abusix.org> writes: > > Hi, > I'm currently experimenting with IMAP using Python 2.7.3 and IMAP4 > from imaplib. I noticed the performance to be very bad. I read 5000 > files from a directory and append them to an IMAP INBOX. The hole > procedure of reading and appending is taking about 210 seconds. > I set up the exact same code in Perl to check if there is a general > IMAP server configuration issue, since CPU and I/O isn't the > problem. The same amount of data on the same IMAP server is done > after 7.9 seconds using Perl. > The difference is huge and I tried to narrow the issue down by > profiling the Python code. > The profile results are, 206 seconds are spent in calling > socket.recv. This just means that most of the time is spent waiting for the server to reply. Perhaps the Perl and Python IMAP libraries use different IMAP commands for appending? Regards Antoine. -- Software development and contracting: http://pro.pitrou.net -- http://mail.python.org/mailman/listinfo/python-list
Re: ssl: add msg_callback
Hi, Thiébaud Weksteen weksteen.fr> writes: > > I wrote a patch for Python 3.2.3 to expose the function > SSL_CTX_set_msg_callback in the module _ssl. > [...] > > Let me know your opinion on that. Does it worth being included? Yes, it sounds useful. Your patch will have to be written against the default branch, although I'm not sure it makes a difference here (see the devguide: http://docs.python.org/devguide/ for more information). Please open an issue at http://bugs.python.org and post your patch there. Thanks Antoine. -- Software development and contracting: http://pro.pitrou.net -- http://mail.python.org/mailman/listinfo/python-list
ANN: pathlib 0.7
Hello, pathlib 0.7 has been released with the following changes: - Add '**' (recursive) patterns to Path.glob(). - Fix openat() support after the API refactoring in Python 3.3 beta1. - Add a *target_is_directory* argument to Path.symlink_to() pathlib offers a set of classes to handle filesystem paths. It offers the following advantages over using string objects: * No more cumbersome use of os and os.path functions. Everything can be done easily through operators, attribute accesses, and method calls. * Embodies the semantics of different path types. For example, comparing Windows paths ignores casing. * Well-defined semantics, eliminating any warts or ambiguities (forward vs. backward slashes, etc.). PyPI download page at http://pypi.python.org/pypi/pathlib/ Documentation at http://pathlib.readthedocs.org/en/latest/ Code and issue tracker at https://bitbucket.org/pitrou/pathlib/ Regards Antoine Pitrou. -- Software development and contracting: http://pro.pitrou.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Flushing buffer on file copy on linux
J gmail.com> writes: > > Now, the problem I have is that linux tends to buffer data writes to a > device, and I want to work around that. When run in normal non-stress > mode, the program is slow enough that the linux buffers flush and put > the file on disk before the hash occurs. However, when run in stress > mode, what I'm finding is that it appears that the files are possibly > being hashed while still in the buffer, before being flushed to disk. Your analysis is partly wrong. It is right that the files can be hashed from in-memory buffers; but even if you flush the buffers to disk using standard techniques (such as fsync()), those buffers still exist in memory, and therefore the file will still be hashed from memory (for obvious efficiency reasons). I don't think there's a portable solution to get away entirely with the in-memory buffers, but under Linux you can write "1" to the special file /proc/sys/vm/drop_caches: $ sudo sh -c "echo 1 > /proc/sys/vm/drop_caches" Or, to quote the /proc man page: /proc/sys/vm/drop_caches (since Linux 2.6.16) Writing to this file causes the kernel to drop clean caches, dentries and inodes from memory, causing that mem‐ ory to become free. To free pagecache, use echo 1 > /proc/sys/vm/drop_caches; tofree dentries and inodes, use echo 2 > /proc/sys/vm/drop_caches; to free pagecache, dentries and inodes, use echo 3 > /proc/sys/vm/drop_caches. Because this is a nondestructive operation and dirty objects are not freeable, the user should run sync(8) first. Regards Antoine. -- Software development and contracting: http://pro.pitrou.net -- http://mail.python.org/mailman/listinfo/python-list
Re: SSLSocket.getpeercert() doesn't return issuer, serial number, etc
Hello, Gustavo Baratto gmail.com> writes: > > SSL.Socket.getpeercert() doesn't return essential information present in the > client certificate (issuer, serial number, not before, etc), and it looks it > is > by design: It does, in Python 3.2: http://docs.python.org/py3k/library/ssl.html#client-side-operation (although the getpeercert() doc should be updated to reflect this) If some information is still lacking from the returned value, please open an issue at http://bugs.python.org Regards Antoine. -- Software development and contracting: http://pro.pitrou.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
Ramchandra Apte gmail.com> writes: > > The zen of python is simply a guideline What's more, the Zen guides the language's design, not its implementation. People who think CPython is a complicated implementation can take a look at PyPy :-) Regards Antoine. -- Software development and contracting: http://pro.pitrou.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
gmail.com> writes: > > Pick up a random text and see the probability this > text match the most optimized case 1 char / 1 byte, > practically never. Funny that you posted a text which does just that: http://mail.python.org/pipermail/python-list/2012-August/629554.html > In a funny way, this is what Python was doing and it > performs better! I honestly suggest you shut up until you have a clue. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for an IPC solution
Laszlo Nagy shopzeus.com> writes: > > There are just so many IPC modules out there. I'm looking for a solution > for developing a new a multi-tier application. The core application will > be running on a single computer, so the IPC should be using shared > memory (or mmap) and have very short response times. But there will be a > tier that will hold application state for clients, and there will be > lots of clients. So that tier needs to go to different computers. E.g. > the same IPC should also be accessed over TCP/IP. Most messages will be > simple data structures, nothing complicated. The ability to run on PyPy > would, and also to run on both Windows and Linux would be a plus. How about the standard multiprocessing module? It supports shared memory, remote processes, and will most probably work under PyPy: http://docs.python.org/library/multiprocessing.html Regards Antoine. -- Software development and contracting: http://pro.pitrou.net -- http://mail.python.org/mailman/listinfo/python-list
Re: The opener parameter of Python 3 open() built-in
Chris Angelico gmail.com> writes: > > On Wed, Sep 5, 2012 at 5:16 AM, Terry Reedy udel.edu> wrote: > > io.open depends on a function the returns an open file descriptor. opener > > exposes that dependency so it can be replaced. > > I skimmed the bug report comments but didn't find an answer to this: > Why not just monkey-patch? When a module function calls on a support > function and you want to change that support function's behaviour, > isn't monkey-patching the most usual? Monkey-patching globals is not thread-safe: other threads will see your modification, which is risky and fragile. Regards Antoine. -- Software development and contracting: http://pro.pitrou.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Using Py_AddPendingCall
css322 gmail.com> writes: > > (1) A worker thread calls Py_AddPendingCall and assigns a handler function. > (2) When the Python interpreter runs, it calls the handler function whenever it yields control to another thread Not exactly. As the documentation says: "If successful, func will be called with the argument arg *at the earliest convenience*." This is a deliberately vague wording to stress that the function will not be called as early as you think. It *should* be called in a timely manner, but not necessarily as soon as the thread switch happens. Which begs the question: > In this example, worker_thread is invoked in C as a pthread worker thread. > This > loops infinitely, but the pending call handler is never invoked. What is your main Python thread doing? Is it running Python code (*)? Or do you have a main Python thread at all? The "main Python thread" is the one from which Py_Initialize() was called. (*) for example, running one of the following functions: http://docs.python.org/dev/c-api/veryhigh.html Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Posix call (execve) breaks mercurial?
Hello, Wayne Werner waynewerner.com> writes: > > So... curiouser and curiouser - it looks like it's not *actually* execve's > fault after all. I just compiled the code from the man page, tweaked it to > run 'hg root', and passed it a new environment. No problems. Well, then I > manually called the posix one from Python and thing worked fine. *Then* I > actually tried the above code, and *it* worked fine. > > However I *still* get problems with the post-review code. So it looks like > when I get back to work on Monday I'll be looking to see what the > difference in environment is there. Your problem is a user question for the Mercurial mailing-list, not a Python problem. See http://selenic.com/mailman/listinfo/mercurial Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: deque and thread-safety
Hello,
Christophe Vandeplas vandeplas.com> writes:
>
> From the documentation I understand that deques are thread-safe:
> > Deques are a generalization of stacks and queues (the name is pronounced
“deck”
> > and is short for “double-ended queue”). Deques support thread-safe, memory
> > efficient appends and pops from either side of the deque with approximately
the
> > same O(1) performance in either direction.
>
> It seems that appending to deques is indeed thread-safe, but not
> iterating over them.
Indeed, if you read the above sentence, deques only support "thread-safe [...]
appends and pops". Other operations are not necessarily thread-safe.
Moreover, any operation which will call back into Python (such as iterating
several times) are *not* thread-safe.
Regardless, deques are not the right container for containment tests
("url in seen"). Like with lists or tuples, a containment test in a deque will
be O(n). So if you want efficient containment tests, you should use a set or a
dict.
Regards
Antoine.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Memory profiling: Python 3.2
Andrew Robinson r3dsolutions.com> writes: > > When Python3.2 is running, is there an easy way within Python to capture > the *total* amount of heap space the program is actually using (eg:real > memory)? I'm not sure what you mean with "real memory" or how precise you want that measurement to be, but you could try http://www.selenic.com/smem/ (and, within Python, re-use smem's concepts, which shouldn't be too difficult) > And how much of that heap space is allocated to variables ( > including re-capturable data not yet GC'd ) ? Again, not sure what you mean with "allocated to variables" (global variables? local variables? everything?). As for "re-capturable data not yet GC'd", the best way to figure out is to run gc.collect() :-) Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Py3k Signal Handling
Jonathan Hayward pobox.com> writes: > > What needs changing here and how should I change it so that handle_signal() > is called and then things keep on ticking? So that it is called when exactly? Your message isn't clear as to what isn't working for you. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Thought of the day
Steven D'Aprano pearwood.info> writes: > > A programmer had a problem, and thought Now he has "I know, I'll solve > two it with threads!" problems. Host: Last week the Royal Festival Hall saw the first performance of a new logfile by one of the world's leading modern programmers, Steven "Two threads" D'Aprano. Mr D'Aprano. D'Aprano: Hello. Host: May I just sidetrack for one moment. This -- what shall I call it -- nickname of yours... D'Aprano: Ah yes. Host: "Two threads". How did you come by it? D'Aprano: Well, I don't use it myself, but some of my friends call me "Two Threads". Host: And do you in fact have two threads? D'Aprano: No, I've only got one. I've had one for some time, but a few years ago I said I was thinking of spawning another, and since then some people have called me "Two Threads". Host: In spite of the fact that you only have one. D'Aprano: Yes. Host: And are you still intending to spawn this second thread? D'Aprano: (impatient) No! Host: ...To bring you in line with your epithet? D'Aprano: No. Host: I see, I see. Well to return to your program. D'Aprano: Ah yes. Host: Did you write this logfile in the thread? D'Aprano: (surprised) No! Host: Have you written any of your recent files in this thread of yours? D'Aprano: No, no, not at all. It's just an ordinary daemon thread. Host: I see, I see. And you're thinking of spawning this second thread to write in! D'Aprano: No, no. Look. This thread business -- it doesn't really matter. The threads aren't important. A few friends call me Two Threads and that's all there is to it. I wish you'd ask me about the logfile. Everybody talks about the threads. They've got it out of proportion -- I'm a programmer. I'm going to get rid of the thread. I'm fed up with it! Host: Then you'll be Steven "No Threads" D'Aprano, eh? -- http://mail.python.org/mailman/listinfo/python-list
Re: nested embedding of interpreter
Hello, Eric Frederich gmail.com> writes: > > 1)Is calling Py_Initialize twice correct, or will I run into other problems > down the road? It's fine in practice (spurious calls are ignored). > I am not sure if there is a mechanism to get something called at the end of the > user's session with the program though, so is it a problem if I don't call > Py_Finalize at the end? Yes, it's a problem. If you don't call Py_Finalize, atexit handlers, object destructors and the like will not be called. These include destructors of file objects (including stdout): any buffering in a still-open file opened for writing can be lost. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple TLS NPN negotiation not working on socket server
Alek Storm gmail.com> writes: > > Connecting with either Firefox 11 or Chrome (which both support NPN) causes > this to print None, rather than a protocol name. What's going on? Ok, I've just tried with Firefox 11. You have to go in "about:config" and set "network.http.spdy.enabled" to true. Then the code snippet works (it prints "spdy/2"). My OpenSSL version is 'OpenSSL 1.0.1-beta3 23 Feb 2012'. I don't have Chrome to test, but perhaps there's a similar configuration option. > Does the protocol matter? PROTOCOL_SSLv23 gives the same result, but PROTOCOL_TLSv1 > makes it die with SSL3_GET_CLIENT_HELLO:wrong version number (strange, because > both browsers ostensibly support TLS). PROTOCOL_TLSv1 works here (with Firefox 11.0). Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Smallest/cheapest possible Python platform?
Roy Smith panix.com> writes: > > What's the smallest/cheapest/lowest-power hardware platform I can run > Python on today? I'm looking for something to use as a hardware > controller in a battery-powered device and want to avoid writing in C > for this project. It depends *which* Python. Complete Python implementations (CPython, PyPy, IronPython, Jython) will have stronger requirements than minimal / incomplete implementations. As for CPython, it needs a C compiler, decent POSIX support, a 32-bit CPU at least, and realistically you won't do much with at least 8 MB RAM. We actually have a buildbot which regularly tests building and running of CPython on an ARM machine: http://www.python.org/dev/buildbot/all/buildslaves/warsaw-ubuntu-arm It's a Cortex A8 with 1GB RAM, though, so I don't know if it's in your range (but 1GB is not needed at all, except that it's nice when running the full regression test suite). Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: [RELEASE] mpdecimal-2.5.0
Hi Stefan On Mon, 29 Jun 2020 15:27:01 +0200 Stefan Krah wrote: > Hi, > > I've released mpdecimal-2.5.0: > >http://www.bytereef.org/mpdecimal/index.html > > 15417edc8e12a57d1d9d75fa7e3f22b158a3b98f44db9d694cfd2acde8dfa0ca > mpdecimal-2.5.0.tar.gz > > Starting with Python 3.9, this version should be used for an external > libmpdec. > > Python versions 3.7 and 3.8 should use the previous version mpdecimal-2.4.2. Speaking of which, is there a plan to incorporate a C API to the _decimal module? Currently, there's no reliable way AFAIK, from C or C++, to take a PyObject* and acccess the underlying `mpd_t*` directly, for fast conversions. Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
parameterized metaclass (or metametaclass)
Hi,
I've been looking at writing parameterized metaclasses and here are the
two solutions I've come to:
(my goal was to build a way to automatically add a hash function that
would take into account a selected list of object attributes)
1. all-in-one metametaclass:
class Autohash2(type):
"""
Metametaclass that instantiates into a metaclass creating
a hash function based on the attributes passed on instantiation.
"""
def __new__(cls, hash_attributes):
def class_new(cls, name, bases, d):
print "New class", name
l = hash_attributes
_hash = hash
_tuple = tuple
c = _hash(_tuple([_hash(k) for k in l]))
def object_hash(obj):
g = obj.__getattribute__
return _hash(_tuple([_hash(g(k)) for k in l]))
d['__hash__'] = object_hash
return super(Autohash2, cls).__new__(cls, name, bases, d)
name = '__private'
bases = (type,)
d = {'__new__': class_new}
print "New metaclass", name
return type.__new__(cls, name, bases, d)
2. with the metametaclass property slightly abstracted away:
class Metametaclass(type):
def __new__(cls, name, bases, dict_):
d = { '__new__': dict_['class_new'] }
def meta_new(cls, *args, **kargs):
print "New metaclass"
name = '__private'
bases = (type,)
return super(Metametaclass, cls).__new__(cls, name, bases, d)
dict_['__new__'] = meta_new
print "New metametaclass", name
return type.__new__(cls, name, bases, dict_)
class Autohash(type):
__metaclass__ = Metametaclass
def __init__(cls, hash_attributes):
cls.hash_attributes = hash_attributes
def class_new(cls, name, bases, d):
print "New class", name
l = cls.hash_attributes
_hash = hash
_tuple = tuple
c = _hash(_tuple([_hash(k) for k in l]))
def object_hash(obj):
g = obj.__getattribute__
return _hash(_tuple([_hash(g(k)) for k in l]))
d['__hash__'] = object_hash
return super(Autohash, cls).__new__(cls, name, bases, d)
Both of those metametaclasses can be used (successfully!) in the
following way:
class Address(object):
__metaclass__ = Autohash3(('host', 'port'))
#
a = Address()
a.host = 'localhost'
a.port =
b = copy.copy(a)
hash(a) == hash(b)
I was wondering if there is some simpler way of building parameterized
metaclasses ?
Regards
Antoine.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Self-defence
Zero Piraeus etiol.net> writes: > > I don't believe that killfiles are a sufficient response in this > situation. > > I can, of course, stop Nikos' posts reaching me, and without too much > hassle also stop replies to his posts reaching me. He would, however, > continue to pollute the list in public, and his posts, whether replied > to or not at the volume he's now sending them, would continue to damage > the reputation of the list and, ultimately, I think possibly kill it. As an occasional reader (and even more occasional poster), I agree with this. As far as I'm concerned, if I were the list owner and if it were easy to ban him, I would already have banned him for a long time. I believe it is not desirable to let poisonous posters do their thing, even if "they might improve". The time spent trying to improve a troll is time not spent caring about and helping other, more benevolent posters, which makes it really harmful to the community (not only pointless) trying to improve a troll. As for the "but banning him would impede free expression" argument, I'd say that people who want a "perfectly free" discussion space can always open their own. We'll see how well it fares in practice, but I'm not holding my hopes very high :-) Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
ANN: pathlib 0.97 released
Hello, As you may know, pathlib has recently been accepted for inclusion into the Python 3.4 standard library. You can view the new module's documentation here: http://docs.python.org/dev/library/pathlib.html As part of the inclusion process, many API changes were done to the original pathlib API (*) so as to satisfy the python-dev community's comments and complaints. Yet, the PyPI version of pathlib (0.8) was still featuring the legacy API. (*) INCLUDING COMPATIBILITY-BREAKING CHANGES I am now releasing version 0.97 of pathlib, which brings the standalone PyPI module up-to-date with the standard library version. The version number reflects this. It isn't yet "officially" stable, since Python 3.4 hasn't been released yet; I plan to do a standalone 1.0 release by the time Python 3.4 is released. If you are not under Python 3.4: Install --- Download from https://pypi.python.org/pypi/pathlib/ or use `pip` or `easy_install`. (be sure to *force* the upgrade if pathlib 0.8 or earlier is already installed!) Documentation - Standalone documentation is at https://pathlib.readthedocs.org/ Changes from 0.8 - The API changes are too long to list; if you are already a pathlib user, I would recommend you read the documentation and find out what has changed for you (your code will likely break loudly, anyway!). Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: Blog "about python 3"
Steven D'Aprano pearwood.info> writes: > > I expect that as excuses for not migrating get fewer, and the deadline for > Python 2.7 end-of-life starts to loom closer, more and more haters^W > Concerned People will whine about the lack of version 2.8 and ask for > *somebody else* to fork Python. > > I find it, hmmm, interesting, that so many of these Concerned People who say > that they're worried about splitting the Python community[1] end up > suggesting that we *split the community* into those who have moved forward > to Python 3 and those who won't. Indeed. This would be extremely destructive (not to mention alienating the people doing *actual* maintenance and enhancements on Python-and-its-stdlib, of which at least 95% are committed to the original plan for 3.x to slowly supercede 2.x). Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: Blog "about python 3"
Hi, Robin Becker reportlab.com> writes: > > For fairly sensible reasons we changed the internal default to use unicode > rather than bytes. After doing all that and making the tests compatible etc etc > I have a version which runs in both and passes all its tests. However, for > whatever reason the python 3.3 version runs slower > > 2.7 Ran 223 tests in 66.578s > > 3.3 Ran 223 tests in 75.703s Running a test suite is a completely broken benchmarking methodology. You should isolate workloads you are interested in and write a benchmark simulating them. Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: "More About Unicode in Python 2 and 3"
On Sun, 05 Jan 2014 08:22:38 -0500 Ned Batchelder wrote: > On 1/5/14 8:14 AM, Mark Lawrence wrote: > > http://lucumr.pocoo.org/2014/1/5/unicode-in-2-and-3/ > > > > Please don't shoot the messenger :) > > > > With all of the talk about py 2 vs. py3 these days, this is the blog > post that I think deserves the most real attention. I haven't had to do > the kind of coding that Armin is talking about, but I've heard more than > one person talk about the difficulty of it in Python 3. > > If anyone wants Python 3 uptake improved, the best thing would be to > either explain to Armin how he missed the easy way to do what he wants > (seems unlikely), or advocate to the core devs why they should change > things to improve this situation. Sometimes the best way to "advocate to the core devs" is to do part of the work, though. There are several people arguing for %-formatting or .format() on bytes, but that still lacks a clear description of which formatting codes would be supported, with which semantics. (see e.g. http://bugs.python.org/issue3982) As for the rest of Armin's rant, well, it's a rant. "In some cases Python 3 is a bit less practical than Python 2" doesn't equate to "Python 3 is broken and 2.8 should be released instead". Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: "More About Unicode in Python 2 and 3"
Ned Batchelder nedbatchelder.com> writes: > > You can look through his problems and decide that he's "wrong," or that > he's "ranting," but that doesn't change the fact that Python 3 is > encountering friction. What happens when a significant fraction of your > customers are "wrong"? Well, yes, there is some friction and this is quite expectable, when shipping incompatible changes. Other pieces of software have undergone a similar process (e.g. Apache 1.x -> Apache 2.x). (the alternative is to maintain a piece of software that sticks with obsolete conventions, e.g. emacs) > Core developers: I thank you for the countless hours you have devoted to > building all of the versions of Python. I'm sure in many ways it's a > thankless task. But you have a problem. What's the point in being > right if you end up with a product that people don't use? People don't use? According to available figures, there are more downloads of Python 3 than downloads of Python 2 (Windows installers, mostly): http://www.python.org/webstats/ The number of Python 3-compatible packages has been showing a constant and healthy increase for years: http://dev.pocoo.org/~gbrandl/py3.html And Dan's survey shows 77% of respondents think Python 3 wasn't a mistake: https://wiki.python.org/moin/2.x-vs-3.x-survey > Maybe there are core developers who are trying hard to solve the > problems Kenneth and Armin are facing. It would be great if that work > was more visible. I don't see it, and apparently Armin doesn't either. While this is being discussed: https://mail.python.org/pipermail/python-dev/2014-January/130923.html I would still point out that "Kenneth and Armin" are not the whole Python community. Your whole argument seems to be that a couple "revered" (!!) individuals should see their complaints taken for granted. I am opposed to rockstarizing the community. Their contribution is always welcome, of course. (as for network programming, the people working on and with asyncio don't seem to find Python 3 terrible) Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: "More About Unicode in Python 2 and 3"
Chris Angelico gmail.com> writes: > > On Tue, Jan 7, 2014 at 3:29 AM, Antoine Pitrou pitrou.net> wrote: > > People don't use? According to available figures, there are more downloads of > > Python 3 than downloads of Python 2 (Windows installers, mostly): > > http://www.python.org/webstats/ > > > > Unfortunately, that has a massive inherent bias, because there are > Python builds available in most Linux distributions - and stats from > those (like Debian's popcon) will be nearly as useless, because a lot > of them will install one or the other (probably 2.x) without waiting > for the user (so either they'll skew in favour of the one installed, > or in favour of the one NOT installed, because that's the only one > that'll be explicitly requested). It's probably fairly accurate for > Windows stats, though, since most people who want Python on Windows > are going to come to python.org for an installer. Agreed, but it's enough to rebut the claim that "people don't use Python 3". More than one million Python 3.3 downloads per month under Windows is a very respectable number (no 2.x release seems to reach that level). Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: "More About Unicode in Python 2 and 3"
Ned Batchelder nedbatchelder.com> writes: > > > I never said they were the whole community, of course. But they are not > outliers either. By your own statistics above, 23% of respondents think > Python 3 was a mistake. Armin and Kenneth are just two very visible > people. Indeed, they are two very visible people. > I'm not creating rock stars. I'm acknowledging that these two people > are listened to by many others. It sounds like part of your effort to > avoid rockstars is to ignore any one person's specific feedback? I must > be misunderstanding what you mean. I am not trying to ignore "any one person's specific feedback". I am ignoring your claim that we should give Armin's blog posts an extraordinary importance because he is "revered". Speaking of which, posting blog articles is not the preferred way to give feedback. There are ample community resources for that. I am irritated that we are apparently supposed to be monitoring blog posts, Twitter feeds and whatnot for any sign of dissent, and immediately react to a criticism that wasn't even voiced directly to us. > You are being given detailed specific feedback from intelligent > dedicated customers that many people listen to, Could you please stop talking about customers? We are not selling Python to anyone (*). Writing open source software as a volunteer is not supposed to be a sacrificial activity where we will bow with extreme diligence to the community's every outburst. Please try to respect us. ((*) Wikipedia: "A customer (sometimes known as a client, buyer, or purchaser) is the recipient of a good, service, product, or idea, obtained from a seller, vendor, or supplier for a monetary or other valuable consideration") Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: "More About Unicode in Python 2 and 3"
Mark Lawrence yahoo.co.uk> writes: > [...] > > And as I started this thread, I'll say what I please, throwing my toys > out of my pram in just the same way that your pal Armin is currently doing. I'll join Ned here: please stop it. You are doing a disservice to everyone. Thanks in advance Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: "More About Unicode in Python 2 and 3"
Terry Reedy udel.edu> writes: > > On 1/6/2014 11:29 AM, Antoine Pitrou wrote: > > > People don't use? According to available figures, there are more downloads of > > Python 3 than downloads of Python 2 (Windows installers, mostly): > > http://www.python.org/webstats/ > > While I would like the claim to be true, I do not see 2 versus 3 > downloads on that page. Did you mean another link? Just click on a recent month, scroll down to the "Total URLs By kB" table, and compute the sum of the largest numbers for each Python version. Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: SQLite + FTS (full text search)
Hi, Mark Summerfield qtrac.plus.com> writes: > > My guess is that on Debian, the packagers install a full SQLite 3 and the Python package uses that. But on > Windows I think the Python packagers bundle their own SQLite (quite rightly since it might not already be installed). > > I'd like the Windows binary to include SQLite 3 with FTS4 support, but I don't know how much work that > involves or if it would make the Python .msi file too big? You can create a feature request on http://bugs.python.org Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: AssertionError (3.X only) when calling Py_Finalize with threads
Hi Tom, Tom Kent gmail.com> writes: > > I'm getting an error output when I call the C-API's Py_Finalize() from a different C-thread than I made a > python call on. Can you please post a bug on https://bugs.python.org ? Be sure to upload your example there. Thank you Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: which async framework?
Chris Withers simplistix.co.uk> writes: > > The protocols are all financial (do we really not have a pure-python FIX > library?!) but none are likely to have existing python implementations. If you are mostly writing protocol implementations (aka parsers and serializers), then you should consider writing them in a way that's framework-agnostic. See as an example: https://pypi.python.org/pypi/obelus/0.1 (if you really want to settle on a single framework and don't mind supporting old Python versions, then I recommend asyncio) Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: which async framework?
Sturla Molden gmail.com> writes: > > Chris Withers simplistix.co.uk> wrote: > > Hi All, > > > > I see python now has a plethora of async frameworks and I need to try > > and pick one to use from: > > > > - asyncio/tulip > > - tornado > > - twisted > > I'd go for using iocp, epoll and kqueue/kevent directly. Why bother to > learn a framework? You will find epoll and kqueue/kevent in the select > module and iocp in pywin32. Yes, why use a library when you can rewrite it all yourself? Actually, you should probably issue system calls to the kernel directly, the libc is overrated (as is portability, I suppose). Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: which async framework?
Sturla Molden gmail.com> writes: > > Antoine Pitrou pitrou.net> wrote: > > > Yes, why use a library when you can rewrite it all yourself? > > This assumes something equivalent to the library will have to be written. > But if it can be replaced with something very minimalistic it is just > bloat. I would also like to respond that the select module and pywin32 are > libraries. Lower-level ones, though. Just like C malloc() is lower-level than Python's memory management. This is the usual assumption that high-level libraries are made of useless cruft piled up by careless programmers. But there are actual reasons why these frameworks have a significant amount of code, and people who decide to ignore those reasons are simply bound to reimplement non-trivial parts of those frameworks in less careful and less tested ways (and they have to maintain it themselves afterwards). What irks me with your response is that you phrased it as though writing a good event loop was an almost trivial thing to do, which it is not once you start considering multiple use cases and constraints. It is quite irresponsible to suggest people don't need the power of network programming frameworks. I would personally distrust a programmer who chooses to reimplement their own event loop, except if they happen to have a very brilliant track record. > Another thing is that co-routines and "yield from" statements just > makes it hard to follow the logic of the program. That's an optional part of Tornado and asyncio, though. It is very reasonable to use Tornado or asyncio and still code in purely callback-driven style (even though Guido himself prefers the coroutine style). Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: What does gc.get_objects() return?
Chris Angelico gmail.com> writes: > > It's not strictly an implementation detail, beyond that there are > certain optimizations. For instance... > > > For CPython 3.4 I guess strings and other atomic types such as ints are > > not, as well as raw object() instances. Custom class instances on the other > > hand seem to be under GC control. > > ... strings and ints should never be listed, and custom objects should > always be listed, but I'd say the non-tracking of object() would be an > implementation-specific optimization. These are all implementation details, tied to the fact that the primary object reclaim mechanism in CPython is reference counting. Other implementations may use a full GC and gc.get_objects() may then also return strings and other "atomic" objects (but the implementation may also elicit to hack get_objects() in order to closely mimick CPython). All in all, though, gc.get_objects() is an expensive function call (it will walk the entire graph of objects tracked by the GC, which can be very large in non-trivial applications), so it's really only useful for debugging (and, I'd add, for low-level debugging). In most situations, gc.get_objects() is certainly the wrong tool to use. Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: Thread._stop() behavior changed in Python 3.4
Hi, Felix Yan gmail.com> writes: > > A minimized snippet to reproduce: > > #!/usr/bin/python > import threading > def stale(): > import time > time.sleep(1000) > t = threading.Thread(target=stale) > t.start() > t._stop() > > This works correctly with Python 3.3, the program exits immediately after > t._stop() called, and no exception was raised. Basically what you are doing is abusing a private method because you want to make the thread daemonic after it was started (a daemonic thread is not waited for at interpreter exit). Please do note one thing: the _stop() method does *not* actually stop the thread; it just marks it stopped, but the underlying OS thread continues to run (and may indeed continue to execute Python code until the interpreter exits). So the obvious "solution" here is to mark the thread daemonic before starting it. A possible related improvement would be to relax the contraints on Thread.daemon to allow setting the flag on a running thread? That said, daemon threads (or abuse of the _stop() method as you did) can lead to instabilities and oddities as some code will continue executing while the interpreter starts shutting down. This has been improved but perhaps not totally solved in recent interpreter versions. A fully correct solution would involve gracefully telling the thread to shut down, via a boolean flag, an Event, a file descriptor or any other means. (if you are interested in this, please open a new issue at http://bugs.python.org) Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
[ANN] pathlib 1.0
Hello, I am announcing the release of pathlib 1.0. This version brings pathlib up to date with the official Python 3.4 release, and also fixes a couple of 2.7-specific issues. Detailed changelog can be found further below. In the future, I expect the standalone (PyPI) version of pathlib to receive little to no updates, except if severe issues are found. New developments and regular bug fixes will happen mostly in the Python standard library, and be publicly available in official Python releases. Overview pathlib offers a set of classes to handle filesystem paths. It offers the following advantages over using string objects: * No more cumbersome use of os and os.path functions. Everything can be done easily through operators, attribute accesses, and method calls. * Embodies the semantics of different path types. For example, comparing Windows paths ignores casing. * Well-defined semantics, eliminating any warts or ambiguities (forward vs. backward slashes, etc.). Requirements Python 3.2 or later is recommended, but pathlib is also usable with Python 2.7. Install --- In Python 3.4, pathlib is now part of the standard library. For Python 3.3 and earlier, ``easy_install pathlib`` or ``pip install pathlib`` should do the trick. Changelog for version 1.0 - - Python issue #20765: Add missing documentation for PurePath.with_name() and PurePath.with_suffix(). - Fix test_mkdir_parents when the working directory has additional bits set (such as the setgid or sticky bits). - Python issue #20111: pathlib.Path.with_suffix() now sanity checks the given suffix. - Python issue #19918: Fix PurePath.relative_to() under Windows. - Python issue #19921: When Path.mkdir() is called with parents=True, any missing parent is created with the default permissions, ignoring the mode argument (mimicking the POSIX "mkdir -p" command). - Python issue #19887: Improve the Path.resolve() algorithm to support certain symlink chains. - Make pathlib usable under Python 2.7 with unicode pathnames (only pure ASCII, though). - Issue #21: fix TypeError under Python 2.7 when using new division. - Add tox support for easier testing. Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: [ANN] pathlib 1.0
Oh, and of course it is published on PyPI: https://pypi.python.org/pypi/pathlib/ Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: imaplib: how to specify SSL/TLS protocol version?
Grant Edwards invalid.invalid> writes: > > Experiments show that when calling ssl.wrap_socket() I have to specify > ssl_version=PROTOCOL_TLSv1 to avoid the above error. > > How do I tell imaplib to use TLS1 instead of SSL3? Use Python 3 and pass the ssl_context parameter to IMAP_SSL: https://docs.python.org/3.3/library/imaplib.html#imaplib.IMAP4_SSL Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: Everything you did not want to know about Unicode in Python 3
Terry Reedy udel.edu> writes: > > On 5/13/2014 8:53 PM, Ethan Furman wrote: > > On 05/13/2014 05:10 PM, Steven D'Aprano wrote: > >> On Tue, 13 May 2014 10:08:42 -0600, Ian Kelly wrote: > >> > >>> Because Python 3 presents stdin and stdout as text streams however, it > >>> makes them more difficult to use with binary data, which is why Armin > >>> sets up all that extra code to make sure his file objects are binary. > >> > >> What surprises me is how hard that is. Surely there's a simpler way to > >> open stdin and stdout in binary mode? If not, there ought to be. > > > > Somebody already posted this: > > > > https://docs.python.org/3/library/sys.html#sys.stdin > > > > which talks about .detach(). > > I sent a message to Armin about this. And the documentation has now been fixed: http://bugs.python.org/issue21364 So something *can* come out of a python-list rantfest, it seems. Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: Pair of filenos read/write each other?
Nobody nowhere.com> writes: > > On Tue, 13 Aug 2013 16:10:41 -0700, Jack Bates wrote: > > > Is there anything like os.pipe() where you can read/write both ends? > > There's socket.socketpair(), but it's only available on Unix. > > Windows doesn't have AF_UNIX sockets, and anonymous pipes (like the ones > created by os.pipe()) aren't bidirectional. I'm not sure I understand the problem: you can just create two pair of pipes using os.pipe(). If that's too low-level, you can wrap the fds using BufferedRWPair: http://docs.python.org/3.3/library/io.html#io.BufferedRWPair (actual incantation would be: r1, w1 = os.pipe() r2, w2 = os.pipe() end1 = io.BufferedRWPair(io.FileIO(r1, 'r'), io.FileIO(w2, 'w')) end2 = io.BufferedRWPair(io.FileIO(r2, 'r'), io.FileIO(w1, 'w')) end1.write(b"foo") end1.flush() end2.read(3) # -> return b"foo" ) An alternative is to use multiprocessing.Pipe(): http://docs.python.org/3.3/library/multiprocessing.html#multiprocessing.Pipe In any case, Python doesn't lack facilities for doing what you want. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pair of filenos read/write each other?
Jack Bates nottheoilrig.com> writes: > > > > An alternative is to use multiprocessing.Pipe(): > > http://docs.python.org/3.3/library/multiprocessing.html#multiprocessing.Pipe > > > > In any case, Python doesn't lack facilities for doing what you want. > > Thank you for your help, I need to satisfy an interface that requires a single > file descriptor number that can be both read from and written to. Is it > possible with any of the solutions you pointed out to get a single file > descriptor number for each end? Yes, it is what multiprocessing.Pipe() provides: http://docs.python.org/3.3/library/multiprocessing.html#multiprocessing.Connection.fileno Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: [RELEASED] Python 3.4.0a2
Le Mon, 9 Sep 2013 08:16:06 -0400, Brett Cannon a écrit : > On Mon, Sep 9, 2013 at 8:02 AM, Larry Hastings > wrote: > > > > > On behalf of the Python development team, I'm chuffed to announce > > the second alpha release of Python 3.4. > > > > This is a preview release, and its use is not recommended for > > production settings. > > > > Python 3.4 includes a range of improvements of the 3.x series, > > including hundreds of small improvements and bug fixes. Major new > > features and changes in the 3.4 release series so far include: > > > > * PEP 435, a standardized "enum" module > > * PEP 442, improved semantics for object finalization > > * PEP 443, adding single-dispatch generic functions to the standard > > library > > * PEP 445, a new C API for implementing custom memory allocators > > * PEP 446, changing file descriptors to not be inherited by default > >in subprocesses > > * PEP 447, a new magic method for metaclasses (``__typelookup__``) > > * PEP 448, making automatic sequence unpacking more general > > > > Those last two PEPs are still in draft form and not accepted nor have > any committed code yet. Unless Larry enthusiastically sneaked them into the release. Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: [python-committers] [RELEASED] Python 3.4.0a2
Le Mon, 9 Sep 2013 14:30:50 +0200, Victor Stinner a écrit : > 2013/9/9 Larry Hastings : > > Python 3.4 includes a range of improvements of the 3.x series, > > including hundreds of small improvements and bug fixes. Major new > > features and changes in the 3.4 release series so far include: > > > > * PEP 446, changing file descriptors to not be inherited by default > >in subprocesses > > The title of the PEP is "Make newly created file descriptors > non-inheritable". It has an impact on all functions creating files and > sockets not only the subprocess module. I don't think Larry's description is wrong. "Non-inheritable" is a shorthand for "non-inheritable in subprocesses" with "subprocesses" taken in the general sense (i.e. not only created with the subprocess module). Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
ANN: Obelus 0.1 -- Asterisk AMI / AGI implementation
Hello,
I'm pleased to announce the first release of Obelus, a MIT-licensed
library to interact with Asterisk using the AMI and AGI protocols.
This is version 0.1, and as such some APIs are a bit draftish and not
guaranteed to be stable accross future releases. Also, documentation is
far from exhaustive.
Quick links
---
* Project page: https://pypi.python.org/pypi/obelus/
* Source code, issue tracker: https://bitbucket.org/optiflowsrd/obelus
* Documentation (incomplete): https://obelus.readthedocs.org
Features
* Python 2 and Python 3 support.
* AMI, FastAGI and Async AGI support.
* Event-driven API friendly towards non-blocking ("async") network
programming styles.
* :pep:`3156`-style protocol implementations.
* Framework-agnostic.
* Adapters for the `Tornado`_, `Twisted`_, `Tulip`_ network programming
frameworks.
* Unit-tested.
Requirements
* Python 2.7, 3.2 or later.
Regards
Antoine.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Tab completion in Python3.4
Steven D'Aprano pearwood.info> writes: > > I don't consider either of these solutions to be satisfactory. If you > agree, I urge you to try it out for yourself, and then leave a comment on > the bug tracker asking for tab completion to still insert tabs at the > beginning of the line: Such a change is much more likely to happen if someone actually proposes a patch for it, rather than merely "ask for it". I don't think anyone is ideologically opposed to it. Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: Python Front-end to GCC
Philip Herron googlemail.com> writes: > > Its interesting a few things come up what about: > > exec and eval. I didn't really have a good answer for this at my talk at PYCon IE 2013 but i am going to say no. I am > not going to implement these. Partly because eval and exec at least to me are mostly from developing > interpreters as a debugging exercise so the test doesn't have to invoke the program properly and feed in > strings to interpret at least thats what i have done in the past with an virtual machine i wrote before gccpy. If you don't implement exec() and eval() then people won't be able to use namedtuples, which are a common datatype factory. As for the rest: well, good luck writing an AOT compiler producing interesting results on average *pure* Python code. It's already been tried a number of times, and has generally failed. Cython mitigates the issue by exposing a superset of Python (including type hints, etc.). Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: Python Front-end to GCC
Steven D'Aprano pearwood.info> writes: > > On Tue, 22 Oct 2013 08:55:15 +, Antoine Pitrou wrote: > > > If you don't implement exec() and eval() then people won't be able to > > use namedtuples, which are a common datatype factory. > > Philip could always supply his own implementation of namedtuple that > doesn't use exec. > > But either way, if he doesn't implement eval and exec, what he has is not > Python, but a subset of Python. Perhaps an interesting and useful subset. If you go that way, we already have Cython (which is both a subset and superset of Python, although I don't know if it's still a strict subset these days). Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: Will Python 3.x ever become the actual standard?
> gmail.com> writes: > > I am starting to have doubts as to whether Python 3.x will ever be actually adopted by the Python community at > large as their standard. We're planning to start the switch on 25th December 2013, 14h UTC. It should be finished at most 48 hours later. You should expect some intermittent problems during the first few hours, but at the end all uses of Twisted will be replaced with Tornado and asyncio (and camelCase methods will have ceased to be). By the way, if you want to join us, one week later we'll also switch the Internet to IPv6 (except Germany). Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
[OT] Re: Python Front-end to GCC
Steven D'Aprano pearwood.info> writes: > > On Fri, 25 Oct 2013 21:36:42 +0100, Mark Lawrence wrote: > > > Mind you, the thought of a bot with a Ph.D. is mind boggling. > > You can buy degrees on the Internet quite cheaply: > > http://en.wikipedia.org/wiki/List_of_animals_with_fraudulent_diplomas > > PhD's are more expensive, which leads me to think that Mark Jenssen is > being a tad flexible with the truth when he claims to have one. He could be an upper-class twit. He would certainly have his chance in the yearly competition ;-) Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
[ANN] pathlib 1.0.1
Hello, I am announcing the release of pathlib 1.0.1. This version makes pathlib Python 2.6-compatible. Note that 2.6 compatibility may not have been as well tested as more recent Python versions (especially on non-Unix platforms). As a reminder, the standalone (PyPI) version of pathlib will not receive any new features or general improvements. New developments and regular bug fixes will happen mostly in the Python standard library, and be publicly available in official Python releases. See https://pypi.python.org/pypi/pathlib Overview pathlib offers a set of classes to handle filesystem paths. It offers the following advantages over using string objects: * No more cumbersome use of os and os.path functions. Everything can be done easily through operators, attribute accesses, and method calls. * Embodies the semantics of different path types. For example, comparing Windows paths ignores casing. * Well-defined semantics, eliminating any warts or ambiguities (forward vs. backward slashes, etc.). Requirements Python 3.2 or later is recommended, but pathlib is also usable with Python 2.6 and 2.7. Install --- In Python 3.4, pathlib is now part of the standard library. For Python 3.3 and earlier, ``easy_install pathlib`` or ``pip install pathlib`` should do the trick. Changelog for version 1.0.1 --- - Pull request #4: Python 2.6 compatibility by eevee. Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: __qualname__ in python 3.3
Hi,
ISE Development gmail.com> writes:
> 'code' object 'function' object
>
> co_name: test __qualname__: test
> co_name: T__qualname__: T
> co_name: method __qualname__: test..T.method
>
> The second call corresponds to the class definition and not the call to the
> constructor (which is in fact a call to 'object.__init__', a C function
> hence not traced as a 'call' event - I checked this by disassembling the
> code object).
There's nothing wrong here. That's just the way things are implemented
internally. This may change in the future without prior notice, so
you shouldn't rely on it.
If you want to dig more, you have to look at how the hidden function ("T")
works:
>>> def f():
... class T: pass
...
>>> f.__code__.co_consts
(None, ", line 2>, 'T')
>>> dis.dis(f.__code__.co_consts[1])
2 0 LOAD_NAME0 (__name__)
3 STORE_NAME 1 (__module__)
6 LOAD_CONST 0 ('f..T')
9 STORE_NAME 2 (__qualname__)
12 LOAD_CONST 1 (None)
15 RETURN_VALUE
Regards
Antoine.
--
https://mail.python.org/mailman/listinfo/python-list
Re: weakref, memory management and execution slow down in PyQt4
kjs riseup.net> writes: > > I have come to believe that the growing number of weakrefs is slowing > down execution. Is my analysis misguided? How can I introspect further? > If the slowdown can be attributed to weakref escalation, what are some > next steps? The way to analyze this is to build some gradually smaller subsets of your application until you can isolate what is causing the growth in number of objects (if any). I would suggest first remove the GUI and replace it with some dummy functions, to stress your core logic. Note that "top" isn't a very reliable tool, as memory fragmentation and other factors can cause your process' visible size to grow even though Python's memory consumption may be stable. There are dedicated Python tools for finer analysis, such as tracemalloc, which is standard on 3.4 and available as a backport for older versions: https://docs.python.org/3/library/tracemalloc.html http://pytracemalloc.readthedocs.org/ But regardless of such tools, the approach above (try to decompose your workload into separate parts until your find the culprit) is highly recommended. Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: Newer Debian versions of python on older Debian distros?
Hi, Chris Angelico gmail.com> writes: > > On Tue, Sep 9, 2014 at 5:04 AM, Travis Griggs gmail.com> wrote: > > Does anyone have experience with using newer versions of python debian packages (in particular, python3 > and python3-bson-ext from ‘testing’) on older stable versions (‘wheezy’ in this case)? If > someone’s figured out how to do this easily, I’d love to hear the recipe! > > > > I don't know about grabbing from testing, because that's fraught with > peril... but there's a pretty easy way to get a newer Python: build > from source. Basically, the standard incantation: [...] I know the OP solved their problem now, but for the record, you can also get binaries of recent Pythons (and other packages) by using conda: http://conda.pydata.org/miniconda.html#miniconda This assumes you don't mind having a local install rather than a system install of Python (something akin to a virtual environment). Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: "High water" Memory fragmentation still a thing?
Hello, Croepha gmail.com> writes: > > Question: > > Does python in general terms (apart from extensions or gc manipulation), exhibit a "high water" type leak of allocated memory in recent python versions (2.7+)? It is not a leak. It is a quite common pattern of memory fragmentation. The article is wrong in assuming that Python doesn't return the memory to the OS. Python does return its empty memory pools to the OS, however the OS itself may not be able to release that memory, because of heap fragmentation. As the article mentions, this was improved (mostly fixed?) in 3.3. Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: "High water" Memory fragmentation still a thing?
Christian Heimes python.org> writes: > > The article doesn't state if the writer is referring to virtual memory > or resident set size. Actually the article mentions the following recipe: resource.getrusage(resource.RUSAGE_SELF).ru_maxrss which means the author is probably looking at resident set size. Regards Antoine. -- https://mail.python.org/mailman/listinfo/python-list
Re: Getting involved
Hello, Sophie Sperner gmail.com> writes: > > Let me ask here please. I'm a first-year PhD student in Ireland. My > background is in mathematics, though I'm going to stream my career > into programming with Python, Java and C++ languages. I have some good > experience with C++ what allowed me to have a presentation in Europe > quite recently - I participated in an open-source cloud computing > project. I used Python a little bit in that project, though mainly it > was in C++. > > [...] > > Could you please list me 2 or 3 projects in Python and/or Java which > are currently active (vivid) and useful? CPython itself - the reference and most used implementation - is written partly in Python (most of the stdlib is) and partly in C. It is easy to find things to do even without touching C. You could take a look: http://docs.python.org/devguide/ Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: IMAP4_SSL and OpenSSL compatibility
W. Martin Borgert debian.org> writes: > > When I add an ssl_version argument to the call to > ssl.wrap_socket() in imaplib.IMAP4_SSL.open(), I can connect to > the Exchange server without problems: > > self.sslobj = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, > ssl_version = ssl.PROTOCOL_SSLv3) > > Would it make sense, to make this change in the Python standard > library? There is already the ssl_context option for that: http://docs.python.org/3.3/library/imaplib.html#imaplib.IMAP4_SSL Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
Steven D'Aprano pearwood.info> writes: > > It is valuable to contrast and compare the PHP and Python docs: > > http://php.net/manual/en/index.php > http://www.python.org/doc/ I suppose you should compare it with http://docs.python.org/3/ instead. > There's no doubt that one of PHP's strengths, perhaps its biggest > strength, is the good state of documentation. My (probably outdated) experience with the PHP docs is that they are very succinct and don't document failure cases or behavioral details at all. Sure, if you only care about the big picture, they are good enough. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
Mitya Sirenef lightbird.net> writes: > I think the issue with python documentation is that it ignores the 95/5 > rule: 95% of people who land on a module's page are only looking for 5% > of its information. The 95/5 rule is generally a fallacy which ignores that the 5% which the readers are expecting to learn about is not the same 5% from reader to reader. (*) Which means that in the end you would really want a diversity of HOWTOs targeted at different usages of the stdlib. But it is a lot of work to write *and* maintain. (*) cf. http://www.joelonsoftware.com/items/2006/12/09.html Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Writing to same file from two threads
Jens Thoms Toerring toerring.de> writes: > > Paul Rubin nospam.invalid> wrote: > > jt toerring.de (Jens Thoms Toerring) writes: > > > in garbled output (i.e. having some output from A inside a > > > line written by B or vice versae) because the "main thread" or > > > Yes they do get garbled like that. Preferred Python style is put a > > single thread in charge of all the i/o to that file, and communicate > > with it by message passing through Queue objects. That is safer than > > directly using locks. > > Thank you for confirmig my suspicion But you have induced > another question: why is using a Queue safer than locking (not > that I doubt that it might be more elegant etc.). Is it "safer" > because it's less likely that one gets it wrong (e.g. by for- > grtting to acquire the lock) or is there something inherently > unsafe about locks? For the record, binary files are thread-safe in Python 3, but text files are not. Locks are safe if you use them well. As you point out, if you forget to acquire your lock, or if you devise a situation where there is a deadlock between competing locks, you can have difficult to diagnose issues. Queues have their internal locking all done for you. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Writing to same file from two threads
Steven D'Aprano pearwood.info> writes: > > On Wed, 27 Feb 2013 13:26:18 +, Antoine Pitrou wrote: > > > For the record, binary files are thread-safe in Python 3, but text files > > are not. > > Where is this documented please? In the documentation, of course ;) http://docs.python.org/3.3/library/io.html#multi-threading Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Speeding up Python's exit
Steven D'Aprano pearwood.info> writes: > > I just quit an interactive session using Python 2.7 on Linux. It took in > excess of twelve minutes to exit, with the load average going well past 9 > for much of that time. > > I think the reason it took so long was that Python was garbage-collecting > a giant dict with 10 million entries, each one containing a list of the > form [1, [2, 3], 4]. But still, that's terribly slow -- ironically, it > took longer to dispose of the dict (12+ minutes) than it took to create > it in the first place (approx 3 minutes, with a maximum load of 4). > > Can anyone explain why this was so painfully slow, and what (if anything) > I can do to avoid it in the future? You are basically asking people to guess where your performance problem comes from, without even providing a snippet so that people can reproduce ;) > I know there is a function os._exit which effectively kills the Python > interpreter dead immediately, without doing any cleanup. What are the > consequences of doing this? I assume that the memory used by the Python > process will be reclaimed by the operating system, but other resources > such as opened files may not be. The OS always disposes of per-process resources when the process terminates (except if the OS is buggy ;-)). However, file buffers will not be flushed, atexit handlers and other destructors will not be called, database transactions will be abandoned (rolled back), etc. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: IMAP4_SSL and OpenSSL compatibility
W. Martin Borgert debian.org> writes:
> >
> > There is already the ssl_context option for that:
> > http://docs.python.org/3.3/library/imaplib.html#imaplib.IMAP4_SSL
>
> Many thanks! Two more questions:
>
> 1. Is there any plan to backport this Python >= 3.3 feature to
> Python 2?
No, we don't backport new features to maintenance releases.
> 2. Would the following lines be correct for Python 3.3?
>
> >>> import imaplib
> >>> IMAP4_SSL("192.168.1.1.", ssl_context =
> SSLContext(ssl.PROTOCOL_SSLv3))
It should be, yes.
Regards
Antoine.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Speeding up Python's exit
Grant Edwards invalid.invalid> writes: > > > I assume that the memory used by the Python process will be reclaimed > > by the operating system, but other resources such as opened files may > > not be. > > All open files (including sockets, pipes, serial ports, etc) will be > flushed (from an OS standpoint) and closed. According to POSIX, no, open files will not be flushed: “The _Exit() and _exit() functions shall not call functions registered with atexit() nor any registered signal handlers. Open streams shall not be flushed. Whether open streams are closed (without flushing) is implementation-defined.” http://pubs.opengroup.org/onlinepubs/9699919799/functions/_exit.html (under the hood, os._exit() calls C _exit()) Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
ANN: pathlib 0.8
pathlib 0.8 has been released at https://pypi.python.org/pypi/pathlib/ Changes --- - Add PurePath.name and PurePath.anchor. - Add Path.owner and Path.group. - Add Path.replace(). - Add Path.as_uri(). - Issue #10: when creating a file with Path.open(), don't set the executable bit. - Issue #11: fix comparisons with non-Path objects. What is pathlib? pathlib offers a set of classes to handle filesystem paths. It offers the following advantages over using string objects: * No more cumbersome use of os and os.path functions. Everything can be done easily through operators, attribute accesses, and method calls. * Embodies the semantics of different path types. For example, comparing Windows paths ignores casing. * Well-defined semantics, eliminating any warts or ambiguities (forward vs. backward slashes, etc.). Requirements Python 3.2 or later is recommended, but pathlib is also usable with Python 2.7. Documentation - The full documentation can be read at `Read the Docs <http://readthedocs.org/docs/pathlib/en/latest/>`_. Contributing The issue tracker and repository are hosted by `BitBucket <https://bitbucket.org/pitrou/pathlib/>`_. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Speeding up Python's exit
Dave Angel davea.name> writes: > > Note he didn't say the python buffers would be flushed. It's the OS > buffers that are flushed. Now please read my message again. The OS buffers are *not* flushed according to POSIX. -- http://mail.python.org/mailman/listinfo/python-list
Re: standalone vs embedded interpreter
Nick Gnedin gmail.com> writes: > I expect it to behave the same way as if I was running it as a > standalone program. On Windows this is indeed the case, but on my Linux > box (Python 3.3.1 (default, Apr 8 2013, 22:33:31) [GCC 4.1.2 20080704 > (Red Hat 4.1.2-51)]) I get a different behavior in handling console > input. A standalone interpreter cycles though the input history when I > use up and down arrows - say, I type this code: > > >>> 1 > 1 > >>> a=4 > >>> a > 4 > > If I now press an key in a standalone interpreter, I get 'a' placed > at the prompt (my previous command). However, in an embedded code I get > > >>> ^[[A > > put at the prompt - does anyone know what I am doing wrong? Are stdin and stdout both interactive? That is, are isatty(fileno(stdin)) and isatty(fileno(stdout)) both true? If you want to debug, take a look at PyOS_Readline() in Parser/myreadline.c. It probably holds the answer to your question. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode issue with Python v3.3
rusi gmail.com> writes: > > Hmm I see some cut-paste goofup on my part. > I was meaning to juxtapose this thread where we put up with inordinate > amount of nonsense from OP > along with the recent thread in which a newcomer who thinks he has > found a bug in pdb is made fun of. > > Then thought better of it and deleted the stuff. > However I did not do a good delete-job so I better now say what I > avoided saying: > > If those who habitually post rubbish are given much of our time and > effort, > whereas newcomers and first-timers are treated rudely, the list begins > to smell like a club of old farts. +1. If you think you have something intelligent to say to jmfauth, you might as well start a private discussion with him. As far as I'm concerned, python-list is *already* of club of old farts. Many regular posters are more interested in "being right on the Internet" rather than helping people out. (this is where the StackOverflow mechanics probably work better, sadly) Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: The node.js Community is Quietly Changing the Face of Open Source
rusi gmail.com> writes: > > Just what I said: ecosystem matters. We may or may not argue about > "more than language", but it surely matters. Some examples: > > 1. In the link that Roderick originally posted there is a long comment > that adds perl to the languages the author discussed. As a language > perl is… um well… its perl. Yet when perl wins its because CPAN > wins. > > 2. Haskell as a language is very well designed. However its package > system -- cabal+hackage -- is completely broken. I think you are deluded. Haskell may very well designed from a language theoretist's point of view, but I suspect most average programmers would find it a hell to code in. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Passing contextual information when logging
Hi Vinay, > I would welcome your views on whether the LoggerAdapter class is > suitable for adding to the logging package in Python 2.6/3.0. Does it > do what might reasonably be expected out of the box? I think it's quite suited to the problem, yes. One question : why does the exception() method call Logger.error() rather than Logger.exception() ? One suggestion : pass in a Logger object rather than a logger name to the LoggerAdapter constructor. (this also means that users could stack LoggerAdapter objects if they wanted to) Thanks Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Moving from Python 2 to Python 3: A 4 page "cheat sheet"
Le Tue, 01 Dec 2009 06:03:36 -0800, Mark Summerfield a écrit : > I've produced a 4 page document that provides a very concise summary of > Python 2<->3 differences plus the most commonly used new Python 3 > features. It is aimed at existing Python 2 programmers who want to start > writing Python 3 programs and want to use Python 3 idioms rather than > those from Python 2 where the idioms differ. [...] > > It is available as a free PDF download (no registration or anything) > from InformIT's website. Here's the direct link: This is great! Just one thing: « Copyright © Qtrac Ltd. 2009. All rights reserved » Might I suggest that you release it under a free license instead? (such as the CC by, CC by-sa, or the Free Art License) Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Recommendation for small, fast, Python based web server
Hello, > I've looked at the web servers that come bundled with the Python > standard library[1] and they are too slow. Apparently you have debugged your speed issue so I suppose you don't have performance problems anymore. Do note, however, that Python is generally not as fast as C -- especially for low-level stuff -- and a Python Web server will probably serve around 10x less requests per second than a C Web server like Apache (this will still give you hundreds of simple requests per second on a modern machine). In any case, as far as functionality, robustness, portability and community support are concerned, you probably can't go wrong with Twisted. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: a huge shared read-only data in parallel accesses -- How? multithreading? multiprocessing?
Le Wed, 09 Dec 2009 06:58:11 -0800, Valery a écrit : > > I have a huge data structure that takes >50% of RAM. My goal is to have > many computational threads (or processes) that can have an efficient > read-access to the huge and complex data structure. > > "Efficient" in particular means "without serialization" and "without > unneeded lockings on read-only data" I was going to suggest memcached but it probably serializes non-atomic types. It doesn't mean it will be slow, though. Serialization implemented in C may well be faster than any "smart" non-serializing scheme implemented in Python. > 2. multi-threading > => d. CPython is told to have problems here because of GIL -- any > comments? What do you call "problems because of the GIL"? It is quite a vague statement, and an answer would depend on your OS, the number of threads you're willing to run, and whether you want to extract throughput from multiple threads or are just concerned about latency. In any case, you have to do some homework and compare the various approaches on your own data, and decide whether the numbers are satisfying to you. > I am a big fan of parallel map() approach I don't see what map() has to do with accessing data. map() is for *processing* of data. In other words, whether or not you use a map()-like primitive does not say anything about how the underlying data should be accessed. -- http://mail.python.org/mailman/listinfo/python-list
Re: Recommendation for small, fast, Python based web server
Le Fri, 11 Dec 2009 19:40:21 +0100, Irmen de Jong a écrit : > > I don't think that number is fair for Python. I think a well written > Python web server can perform in the same ballpark as most mainstream > web servers written in C. Especially Apache, which really isn't a top > performer. And I'm pretty sure a well written Python server can > outperform a badly written C based server easily. The order of magnitude I gave is based on real-world testing. You are under-estimating how much of an impact Python's interpretation speed has on low-level code. Even Apache *is* a top performer compared to Python web servers. -- http://mail.python.org/mailman/listinfo/python-list
Re: Dangerous behavior of list(generator)
Le Mon, 14 Dec 2009 15:21:09 +, exarkun a écrit : > > I'm asking about why the behavior of a StopIteration exception being > handled from the `expression` of a generator expression to mean "stop > the loop" is accepted by "the devs" as acceptable. It's not "accepted as acceptable", it's just a side effect of how various means of iterating (including for loops and generators) are implemented in CPython. Seeing how it doesn't seem to prevent or promote any useful programming idiom, there was no incentive to either 1) codify it as official spec or 2) change it. In other words, it should be considered undefined behaviour, and perhaps other Python implementations behave differently. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Where is PyMethod_GET_CLASS in Python 3?
Le Tue, 15 Dec 2009 08:08:01 -0800, Infinity77 a écrit : > > When building C extensions In Python 2.X, there was a magical > PyMethod_GET_CLASS implemented like this: > > #define PyMethod_GET_CLASS(meth) \ > (((PyMethodObject *)meth) -> im_class) > > It looks like Python 3 has wiped out the "im_class" attribute. Which is > the alternative was to handle this case in Python 3? First, is it a bound method? Unbound methods are just function objects in py3k. Check that PyMethod_Check() returns true. Second, have you tried Py_TYPE(PyMethod_GET_SELF(meth))? > BTW, it's very, very, > *very* hard to find any possible reference to help migrating existing C > extensions from Python 2.X to Python 3. There's http://docs.python.org/3.1/howto/cporting.html You are encouraged to post any suggestions or corrections on the bug tracker: http://bugs.python.org Finally, there's also a dedicated mailing-list for porting to py3k: http://mail.python.org/mailman/listinfo/python-porting While it hasn't seen a lot of activity lately, I'm sure there are people there willing to answer any questions you have! Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python3.1: gzip encoding with UTF-8 fails
Hello,
Le Sun, 20 Dec 2009 17:08:33 +0100, Johannes Bauer a écrit :
>
> #!/usr/bin/python3
> import gzip
> x = gzip.open("testdatei", "wb")
> x.write("ä")
The bug here is that you are trying to write an unicode text string ("ä")
to a binary file (a gzip file). This bug has been fixed now; in the next
3.x versions it will raise a TypeError:
>>> x = gzip.open("testdatei", "wb")
>>> x.write("ä")
Traceback (most recent call last):
File "", line 1, in
File "/home/antoine/py3k/__svn__/Lib/gzip.py", line 227, in write
self.crc = zlib.crc32(data, self.crc) & 0x
TypeError: must be bytes or buffer, not str
You have to encode manually if you want to write text strings to a gzip
file:
>>> x = gzip.open("testdatei", "wb")
>>> x.write("ä".encode('utf8'))
Regards
Antoine.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Semantic of operations on a closed file object
Hello,
> But the io.IOBase.close() method document says: """Once the file is
> closed, any operation on the file (e.g. reading or writing) will raise
> an IOError .""" which unlike the
> class doc is not conditional about the behavior...
>
> Experimentation (see below) show that I get a ValueError in practice
> (python 3.1) with io.BufferedWriter and io.StringIO objects.
>
> So which one is right? Am I reading the wrong documentation?
The documentation is wrong in this case. ValueError is indeed returned,
as is already the case in Python 2.x:
Python 2.6.4 (r264:75706, Oct 27 2009, 15:50:27)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> f =open('foo', 'w')
>>> f.close()
>>> f.write('a')
Traceback (most recent call last):
File "", line 1, in
ValueError: I/O operation on closed file
You could/should open a bug about this on http://bugs.python.org so that
it isn't forgotten.
Thank you
Antoine.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Recommendation for small, fast, Python based web server
Le Fri, 25 Dec 2009 10:38:19 -0800, Aahz a écrit : > In article , > Antoine Pitrou wrote: >> >>Apparently you have debugged your speed issue so I suppose you don't >>have performance problems anymore. Do note, however, that Python is >>generally not as fast as C -- especially for low-level stuff -- and a >>Python Web server will probably serve around 10x less requests per >>second than a C Web server like Apache (this will still give you >>hundreds of simple requests per second on a modern machine). > > For static pages or dynamic pages? Once you get into dynamic pages, I > sincerely doubt that the smaller Apache overhead makes lots of > difference. For static pages. Yes, I guess the difference for dynamic pages would indeed be quite small. Especially if database activity is also involved. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: detect interactivity
Le Tue, 29 Dec 2009 16:09:58 +0100, Roald de Vries a écrit : > Dear all, > > Is it possible for a Python script to detect whether it is running > interactively? It can be useful for e.g. defining functions that are > only useful in interactive mode. Try the isatty() method (*) on e.g. stdin: $ python -c "import sys; print sys.stdin.isatty()" True $ echo "" | python -c "import sys; print sys.stdin.isatty()" False (*) http://docs.python.org/library/stdtypes.html#file.isatty -- http://mail.python.org/mailman/listinfo/python-list
Re: detect interactivity
Le Wed, 30 Dec 2009 03:33:18 +0100, Roald de Vries a écrit : > > I'm using a database, and want to use python interactively to manipulate > it. On the other hand, I also want to be able to use it non- > interactively. In that case, it would be a waste of CPU to load the > function/class definitions meant for interactive use. Why don't you write a separate writer script for the interactive behaviour? This would be much simpler than trying to guess whether the current Python process is "interactive", since the latter is ill-defined. Remember, explicit is better than implicit. -- http://mail.python.org/mailman/listinfo/python-list
Re: multithreading, performance, again...
> 39123 function calls (38988 primitive calls) in 6.004 CPU > seconds > [...] > > It's not burning CPU time in the main thread (profiling with cProfile > indicated smth similar to the above), it's not burning it in the > individual worker threads What do you mean, it's not burning CPU time? The profile output above seems to suggest 6 CPU seconds were "burnt". By the way, I don't think running several concurrent profile sessions dumping *to the same stats file* is supported, consider using a separate stats file for each thread or the results may very well be bogus. -- http://mail.python.org/mailman/listinfo/python-list
Re: fsync() doesn't work as advertised?
Le Mon, 04 Jan 2010 08:09:56 -0800, Brian D a écrit : > > What I've seen is that flush() alone produces a complete log when the > loop finishes. When I used fsync(), I lost all of the write entries > except the first, along with odd error trap and the last entry. Perhaps you are writing to the file from several threads or processes at once? By the way, you shouldn't need fsync() if you merely want to look at the log files, because your OS will have an up-to-date view of the file contents anyway. fsync() is useful if you want to be sure the data has been written to the hard disk drive, rather than just kept in the operating system's filesystem cache. -- http://mail.python.org/mailman/listinfo/python-list
Re: Speeding up network access: threading?
Le Tue, 05 Jan 2010 15:04:56 +0100, Jens Müller a écrit : > > Is a list thrad-safe or do I need to lock when adding the results of my > worker threads to a list? The order of the elements in the list does not > matter. The built-in list type is thread-safe, but is doesn't provide the waiting features that queue.Queue provides. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: please help shrink this each_with_index() implementation
>> Couldn't you just use the built-in enumerate() to replace the whole >> thing? > > Because that would involve, like, reading an entire Python book just to > locate that method? Actually, no. It just involves reading one of the most important pages in the documentation, the page which describes the built-in functions: http://docs.python.org/library/functions.html Don't forget that the Python documentation is rich and structured. And good luck. -- http://mail.python.org/mailman/listinfo/python-list
Re: please help shrink this each_with_index() implementation
> The point: int('') or int('something') both throw an error. In general,
> this is hand-holding, but in specific I don't think the "rich and
> structured" documentation will cover how to beat a 0 out of it in less
> than 3 lines.
Because it's a bad idea to do so and Python doesn't encourage such sloppy
behaviour. If you like it, though, you might prefer PHP.
By the way: error reporting is definitely *not* hand-holding. It is
simply not good to let errors pass silently by default. Again, if you
think the contrary, PHP is your friend ;-)
--
http://mail.python.org/mailman/listinfo/python-list
