Re: Guido sees the light: PEP 8 updated
On 2016-04-18 21:14:02 +, Pete Forman said: Why is it that Python continues to use a fixed width font and therefore specifies the maximum line width as a character count? An essential part of the language is indentation which ought to continue to mandate that lines start with a multiple of 4 em worth of space (or some other size or encode with hard tabs, that is not germane to my question). The content of the line need not be bound by the rules needed to position its start. I wrote a semi-serious, somewhat tongue-in-cheek article entitled "Your code style guide is crap, but still better than nothing." a number of years ago, and reposted it towards the end of last year. It would seem to apply here, as the fundamental disconnect isn't just "use of N space characters", but "use of space characters at all". From the article: Do you use spaces in a word processor to line up bullet points? If you do you’ll be first against the wall when the revolution comes! http://s.webcore.io/2K0W0m2T2e2f It also touches on points raised by others, such as Elastic Tabstops. — Alice. -- https://mail.python.org/mailman/listinfo/python-list
Re: Needed: Real-world examples for Python's Cooperative Multiple Inheritance
On 2010-11-24 12:08:04 -0800, Raymond Hettinger said: I'm writing-up more guidance on how to use super() and would like to point at some real-world Python examples of cooperative multiple inheritance. The SocketServer module (http://docs.python.org/library/socketserver.html) uses cooperative multiple inheritance to implement threading / async using a ThreadingMixIn class and multi-processing using a ForkingMixIn class, which may not be as complicated a use case as you are looking for. One thing that caught me up was the attribute resolution order; it's a FIFO, with the first superclass being examined preferentially over later superclasses in the declaration. (Mixins go before the class they extend.) — Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: building a web interface
Howdy! I'm mildly biased, being the author of the framework, but I can highly recommend WebCore for rapid prototyping of web applications; it has templating via numerous template engines, excellent JSON (AJAJ) support, and support for database back-ends via SQLAlchemy. It also has session support baked-in via a project called Beaker. Documentation is fairly complete, and I can be found camping in the #webcore IRC channel on irc.freenode.net at strange hours. If you can write a class, you can have a fully operational web application in a single file of ~8 lines or so. (Or you can create a complete easy-installable Python package with multiple modules.) For information, see: http://www.web-core.org/ As an interactive-fiction example: class RootController(web.core.Controller): def index(self): """This returns a template that uses JavaScript to call execute(). The JavaScript adds the result of execute() to the display.""" session = db.Session().save() return './templates/main.html', dict(session=session.id) def execute(self, session, statement): """Load our session and pass the input off to our interactive fiction library of choice. Return the result if all went well.""" session = db.Session.get(session) try: result = myiflib.execute(session, statement) except myiflib.ParseError: return 'json:', dict(status="failure", message="Error...") return 'json:', dict(status="success", message=result) — Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Processing file with lists.
You describe a two-part problem. The first, loading the data, is
easily accomplished with the Python CSV module:
http://docs.python.org/library/csv.html
e.g.: reader = csv.reader(open('filename', 'rb'), delimiter=';',
quotechar=None)
In the above example, you can iterate over 'reader' in a for loop to
read out each row. The values will be returned in a list.
You could also use a DictReader to make the data more naturally
accessible using name=value pairs.
I want to know how could I process this file using ' lists ' ,
that could answer questions like . How many ? , Who did .. ?
etc.
This isn't very clear, but if your dataset is small (< 1000 rows or so)
you can fairly quickly read the data into RAM then run through the data
with loops designed to pull out certain data, though it seems your data
would need additional processing. (The authorship information should
be split into two separate columns, for example.)
An alternative would be to load the data into a relational database
like MySQL or even SQLite (which offers in-memory databases), or an
object database such as MongoDB which supports advanced querying using
map/reduce.
You'd have to examine the documentation on these different systems to
see which would best fit your use case. I prefer Mongo as it is very
easy to get data into and out of, supports SQL-like queries, and
map/reduce is extremely powerful.
— Alice.
--
http://mail.python.org/mailman/listinfo/python-list
Re: a regexp riddle: re.search(r'(?:(\w+), |and (\w+))+', 'whatever a, bbb, and c') =? ('a', 'bbb', 'c')
Accepting input from a human is frought with dangers and edge cases. ;) Some time ago I wrote a regular expression generator that creates regexen that can parse arbitrarily delimited text, supports quoting (to avoid accidentally separating two elements that should be treated as one), and works in both directions (text<->native). The code that generates the regex is heavily commented: https://github.com/pulp/marrow.util/blob/master/marrow/util/convert.py#L123-234 You should be able to use this as-is and simply handle the optional 'and' on the last element yourself. You can even create an instance of the class with the options you want then get the generated regular expression by running print(parser.pattern). Note that I have friends who use 'and' multiple times when describing lists of things. :P — Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: a regexp riddle: re.search(r'(?:(\w+), |and (\w+))+', 'whatever a, bbb, and c') =? ('a', 'bbb', 'c')
Now that I think about it, and can be stripped using a callback
function as the 'normalize' argument to my KeywordProcessor class:
def normalize(value):
value = value.strip()
if value.startswith("and"):
value = value[3:]
return value
parser = KeywordProcessor(',', normalize=normalize, result=list)
— Alice.
--
http://mail.python.org/mailman/listinfo/python-list
Re: do something every n seconds
how can I do something (e.g. check if new files are in the working directory) every n seconds in Python? The simplest method is executing time.sleep(n) within an infinite while loop. There are more elegant solutions: using coroutine frameworks, threaded task schedulers, etc. — Alice. -- http://mail.python.org/mailman/listinfo/python-list
Strategies for unit testing an HTTP server.
Hello! Two things are missing from the web server I've been developing before I can release 1.0: unit tests and documentation. Documentation being entirely my problem, I've run into a bit of a snag with unit testing; just how would you go about it? Specifically, I need to test things like HTTP/1.0 request/response cycles, HTTP/1.0 keep-alives, HTTP/1.1 pipelined reuqest/response cycles, HTTP/1.1 connection: close, HTTP/1.1 chunked requests/responses, etc. What is the recommended / best way to test a daemon in Python? (Note that some of the tests need to keep the "client" socket open.) Is there an easy way to run a daemon in another thread, then kill it after running some tests across it? Even better, are there any dedicated (Python or non-Python) HTTP/1.1 compliance testing suites? Thank you for any assistance, — Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: how to go on learning python
Howdy Xavier!
[Apologies for the length of this; I didn't expect to write so much!]
I've been a Python programmer for many years now (having come from a
PHP, Perl, C, and Pascal background) and I'm constantly learning new
idioms and ways of doing things that are more "Pythonic"; cleaner, more
efficient, or simply more beautiful. I learn by coding, rather than by
reading books, taking lectures, or sitting idly watching screencasts.
I constantly try to break the problems I come up with in my head into
smaller and smaller pieces, then write the software for those pieces in
as elegant a method as possible.
Because of my "turtles all the way down" design philosophy, a lot of my
spare time projects have no immediate demonstrable benefit; I code them
for fun! I have a folder full of hundreds of these little projects,
the vast majority of which never see a public release. I also collect
little snippets of code that I come across[1] or write, and often
experiment with performance tests[2] of small Python snippets.
Often I'll assign myself the task of doing something far outside my
comfort zone; a recent example is writing a HTTP/1.1 web server. I had
no idea how to do low-level socket programming in Python, let alone how
HTTP actually worked under-the-hood, and because my goal wasn't
(originally) to produce a production-quality product for others it gave
me the freedom to experiment, rewrite, and break things in as many ways
as I wanted. :) I had people trying to convince me that I shouldn't
re-invent the wheel ("just use Twisted!") though they mis-understood
the reason for my re-invention: to learn.
It started as a toy 20-line script to dump a static HTTP/1.0 response
on each request and has grown into a ~270 line fully HTTP/1.1
compliant, ultra-performant multi-process HTTP server rivalling pretty
much every other pure-Python web server I've tested. (I still don't
consider it production ready, though.) Progressive enhancement as I
came up with and implemented ideas meant that sometimes I had to
rewrite it from scratch, but I'm quite proud of the result and have
learned far more than I expected in the process.
While I don't necessarily study books on Python, I did reference HTTP:
The Definitive Guide and many websites in developing that server, and I
often use the Python Quick Reference[3] when I zone out and forget
something basic or need to find something more advanced.
In terms of understanding how Python works, or how you can use certain
semantics (or even better, why you'd want to!) Python Enhancement
Proposals (PEPs) can be an invaluable resource. For example, PEP
318[4] defines what a decorator is, why they're useful, how they work,
and how you can write your own. Pretty much everything built into
Python after Python 2.0 was first described, reasoned, and discussed in
a PEP.
If you haven't seen this already, the Zen of Python[5] (a PEP) has many
great guidelines. I try to live and breathe the Zen.
So that's my story: how I learn to improve my own code. My motto,
"re-inventing the wheel, every time," is the short version of the
above. Of course, for commercial work I don't generally spend so much
time on the nitty-gritty details; existing libraries are there for a
reason, and, most of the time, Getting Things Done™ is more important
than linguistic purity! ;)
— Alice.
[1] https://github.com/GothAlice/Random/
[2] https://gist.github.com/405354
[3] http://rgruet.free.fr/PQR26/PQR2.6.html
[4] http://www.python.org/dev/peps/pep-0318/
[5] http://www.python.org/dev/peps/pep-0020/
--
http://mail.python.org/mailman/listinfo/python-list
Re: Comparison with False - something I don't understand
Howdy! When I run pychecker through my modules I get the message that comparisons with "False" is not necessary and that it might yield unexpected results. Comparisons against False -are- dangerous, demonstrated below. Yet in some situations I need to specifically check whether False was returned or None was returned. Why is comparison with False so bad? (False == 0) is True (True == 1) is True The bool type is a subclass of int! (Run those lines in a Python interpreter to see. ;) if var == False: if var is False: … So how do you get around this? My functions return False and None under different circumstances. Should I raise exceptions instead? I feel it's unnecessary clutter to use exceptions unless absolutely no other solution is available and yet I have doubts about the "False" value. If you want to check not just for value equivelance (False == 0) but literal type, use the "is" comparator. "is" checks, and others correct me if I'm wrong, the literal memory address of an object against another. E.g. False, being a singleton, will always have the same memory address. (This is true of CPython, possibly not of Python implementations like Jython or IronPython.) Using "is" will be effective for checking for literal None as well. When ever I need to test for None, I always use the "is" comparator. It's also more English-like. (None, evaluating to False when using '==', is useful when all you care about is having a blank default value, for example.) — Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: A web site using Python
1. Pick a web framework, I'd suggest looking at: Django (http://www.djangoproject.com/) Pyramid (http://docs.pylonshq.com/pyramid/dev/) I'm biased, but I can highly recommend WebCore (http://www.web-core.org/) as it more easily supports small to mid-sized applications and actively encourages the use of standard Python idioms. A reasonable example (though it was hurried) would be the codebase behind tsatimeline.org: https://github.com/GothAlice/TSA-Timeline The important files are application.py (controllers), model.py (data model), and the templates (views) folder. (The public folder is where CSS/JS/images go.) Similar to Stef's comment on web2py, development with WebCore (or web2py, or WebPy, or… basically any micro-framework) is extremely rapid. — Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Average Salary Report
Howdy! I'm doing a market research report on the average hourly rates for Python/Django developers. Any input on this? I do understand that it does depend on the location, amount of experience and skills. I'd like to hear what are the hourly rates within your area :-) Thanks! I'm a strange case, but I like to charge my clients what my clients charge theirs. It's only fair. ;) This, of course, results in my rates varying from between $25 and $125, with a mean of ~$50/hr. More often, though, fixed payment is arranged on a per-project basis regardless of hours, which I understand to be a dangerous practice. (Scope creep is my arch nemesis! ;) - Alice -- http://mail.python.org/mailman/listinfo/python-list
Re: Design Ideals Goals Python 3 - Forest for the trees
I was interested in what the design goals/philosphy was of Python 3 from a birds eye view, forest for the trees approach. I think I can safely point to the Zen of Python[1] as many of the points therein directly apply to the simplifiation, clarification, and goals of Python 3. Most notably: :: Beautiful is better than ugly. E.g. dict.iteritems, dict.iterkeys, dict.itervalues? Strip 'iter' and it's fixed. :: Special cases aren't special enough to break the rules. Ever get hung up on core Python modules with title caps? Yeah, those are fixed. :: There should be one-- and preferably only one --obvious way to do it. E.g. urllib, urllib2, urllibX… yeah, that's been fixed, too. :) :: Namespaces are one honking great idea -- let's do more of those! Numerous modules have been merged, or moved into more manageable (and logical) namespaces. I can safely assume one goal was speed improvement as in the blog he noted "Don’t fret too much about performance--plan to optimize later when needed." So I assume that means that Python had developed to a point where that was needed. The Python GIL (Global Interpreter Lock) has been getting a lot of negative attention over the last little while, and was recently fixed to be far more intelligent (and efficient) in Python 3.2. There are numerous other performance improvements, for which yo ucan examine the change logs. But performance wouldn't be the over-arching criteria for the change. Just curious. Clarification, simplification, specivity, efficiency, … just be more "Pythonic". Note that I'm not a core Python contributor or have ever communicated with the BDFL: this is just the viewpoint of somoene doing her darnd'est to encourage Python 3 support. ;) All of the new projects I work on are Python 2.6+ and Python 3.1+ compatible. (Arguments against dual-compatible polygot code can go to /dev/null for the purposes of this thread.) - Alice [1] http://www.python.org/dev/peps/pep-0020/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Design Ideals Goals Python 3 - Forest for the trees
So do the new changes(to the GIL) nullify concerns raised by David Beazely here http://dabeaz.com/python/UnderstandingGIL.pdf Ah, good catch. I had watched the recorded presentation some time ago. Yes, the rewritten GIL should alleviate those problems. Instead of simply releasing and re-acquiring the GIL, it releases, then determines thread scheduling using the brainfuck algorithm instead of leaving it up to the kernel in the brief instant the GIL is unassigned (which often doesn't context switch to another thread, thus the performance penalty). (I beleive that algorithm, whose name -is- accurate, was the winner of the long, long discussion on the Python ticket.) Some projects have been using and requiring psyco to gain speed improvements in python http://psyco.sourceforge.net/introduction.html however it seems that the developer is not active on this project anymore and is more active on PyPy http://codespeak.net/pypy/dist/pypy/doc/ I've never really attempted to use JIT optimizers due to the fact that all of my development and production environments are 64-bit, and I haven't found one yet that supports 64-bit properly. Relying on dead projects, however, is an issue of larger scope than mere portability. ;) A program such as AVSP http://avisynth.org/qwerpoi/ which relies on psyco what would be a good proposition to use when taking the project to python 3.0 if psyco will remain unavailable? I'd take the same approach Python 3 itself did; rewrite it for Python 3 and take the opportunity to remove excessive backwards compatibility cruft, streamline algorithms, etc. With a suite of existing unit/functional tests, that possibility is the ultimate in test-driven development. ;) It would also follow the write clean code, then profile and optimize where actually needed philosophy. Obviously that recommendation won't be the "best" solution for every project. With all of the FOSS projects I really, really care about I'm writing from near-scratch (the code, if not the algorithms) with 2.6+ and 3.1+ polygot (no conversion tools like 2to3, and no split packaging) compatibility as a primary design goal. So far it's working out quite well. - Alice -- http://mail.python.org/mailman/listinfo/python-list
Re: Python3 Web Framework
On 2010-12-30 23:47:17 -0800, Aman said: Hey all... I just started with Python, and I chose Python3 because it seemed a subtle choice as compared to doing Pthon 2.x now and then porting to Python3.x later... I plan to start with Web Development soon... I wanted to know what all web frameworks are available for Python3... I heard the Django is still not compatible with 3.x... Any idea guys? Python 3 has a number of issues with web development thus far: WSGI[1] (PEP 333) isn't directly compatible with Python 3, for one. However, PEP is looking good[2] for making web framework code compatible with Python 3 without needing too much modification. I'm not sure what the state of affairs is for PEP or Python 3 compatible frameworks, however. (CherryPy -might- be compatible, I can not recall.) Basically this means that using Python 3, you'll be "roughing it" for a while. On the other hand, I'm working on PEP 444[3] (WSGI 2) and have a highly performant web server compatible with Python 3 available[4] that is compatible with PEP 444 as published on Python.org[5] (master branch) and with my rewritten draft (draft branch, to be merged when my rewrite is complete and published on Python.org). Another developer and I have been working on the WebOb-style helper exceptions and wrappers, from which a microframework can quickly spawn. The HTTP server has 100% coverage (master) and near-100% coverage (draft) and compatibility with Python 2.6+ and 3.1+. Have a great day, - Alice. [1] http://www.python.org/dev/peps/pep-0333/ [2] http://www.python.org/dev/peps/pep-/ [3] http://bit.ly/e7rtI6 [4] http://bit.ly/fLfamO [5] http://bit.ly/gmT17O -- http://mail.python.org/mailman/listinfo/python-list
Re: Python3 Web Framework
On 2010-12-31 02:20:47 -0800, Terry Reedy said: I believe some will be improved or even solved in 3.2. Evidence to back this statement up would be appreciated. ;) - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Interrput a thread
On 2010-12-31 10:28:26 -0800, John Nagle said: Even worse, sending control-C to a multi-thread program is unreliable in CPython. See "http://blip.tv/file/2232410"; for why. It's painful. AFIK, that has been resolved in Python 3.2 with the introduction of an intelligent thread scheduler as part of the GIL release/acquire process. - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: tempfile.NamedTemporaryFile use case?
I'm a bad person, but one use case I have is for shuffling templates
around such that:
* An inherited ('parent') template can be stored in a database.
* The 'views' of my application are told to either use the real master
template or the db parent template.
* The rendering engine loads the parent from disk.
Thus I create a NamedTemporaryFile (with some custom prefix and suffix
stuff), write the parent template from DB to it, flush, then let the
rendering engine do its thing. Works like a hot damn, and lets the
users of my CMS manage custom layouts from within the CMS.
Templates are cached using the CMS template path as the dict key, and a
2-tuple of modification time and the NamedTemporaryFile as the value.
Cleanup of old versions on-disk is simple: just close the file!
- Alice.
--
http://mail.python.org/mailman/listinfo/python-list
Re: opinion: comp lang docs style
On 2011-01-04 22:29:31 -0800, Steven D'Aprano said: In any case, your assumption that any one documentation work should stand on its own merits is nonsense -- *nothing* stands alone. +1 How many RFCs still in use today don't start with: The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 I posted a response on the article itself, rather than pollute a mailing list with replies to a troll. The name calling was a rather large hint as to the intention of the "opinion", either that or whoever translated the article (man or machine) was really angry at the time. ;) - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Trying to decide between PHP and Python
On 2011-01-04 12:53:02 -0800, Alan Meyer said: I confess that I haven't used PHP so someone correct me if I'm wrong. [snip] +1 You're pretty much on the ball with your description. I might summarize it as: PHP (PHP: Hypertext Processor) is a templating language with a significant enough standard library for aspirations of general-purpose scripting. Python is a general-purpose scripting language with numerous templating languages, let alone the terrifyingly large standard library. ;) Yes, PHP has templating language syntaxes like Smarty, but you're running a template parser within another template parser... A fairly common sarcastic quote is: "You aren't a real Python programmer until you write your own [coroutine framework | web framework | templating language | ...]!" I've done all three, and this probably does not make me a good person. ;) The coroutine framework was a hack to see how they work through experimentation, but I'm quite proud of the web framework and templating system! :D - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Streaming templating languages for use as WSGI body.
Howdy! I'm trying to find a templating engine whose templates can be consumed directly as a WSGI response body iterable. So far I haven't been very successful with Google; the engines I've found universally generate a monolithic rendered string. Bonus points for templating engines that support flush() mechanics internally to allow developer-chosen 'break points' in the iterable. Bonus points++ if it can somehow calculate the response body length without generating the whole thing monolithically. ;) - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Streaming templating languages for use as WSGI body.
Not sure if it's bad form to respond to your own posts, but here goes. ;) Coding up a quick hack of a templating engine, I've produced this: http://pastie.textmate.org/private/ws5jbeh1xyeaqtrhahevqw (The implementation of the engine itself is a base class that overrides __call__ and __getitem__, with __unicode__ serializing in one block, the norm for templating engines, and .render(encoding='ascii') returning a generator using cStringIO for internal buffering.) I even have a light-weight widget system based on it, now. E.g. class Input(Widget): type_ = None @property def template(self): return tag.input ( type_ = self.type_, name = self.name, id = self.name + '-field', value = self.value, **self.args ) I'll polish it and package it up. :) - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching Python-list
On 2011-01-05 17:31:13 -0800, Slie said: I was wondering if anyone could tell me how to search through the Archives otter then manually looking through each month. Grab a Usenet news reader (such as Thunderbird or Unison), point it at: nntps://news.gmane.org/gmane.comp.python.general I can rather quickly search through articles using this interface (let alone keep my e-mail clear of mailing lists) and even post. :) - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: How suitable is Python to write system utilities?
On 2011-01-06 01:35:58 -0800, Rohit Coder said: Is Python suitable to write low-level system utilities like Defrag, Malware Removal Tools and Drivers? Yes and no. Python does include libraries (and has available third-party libraries) to interface with external low-level libraries of every kind, has Python-native third-party libraries to do things like examine ELF object files / executables, manipulate raw IP datagrams, etc, and it is possible to access glib (and other C libraries or even Windows DLLs) directly from within Python without creating wrapper interfaces. While you -can- do all of these things, the question becomes do you really -want to-? I've implemented server monitoring suites, FUSE filesystem drivers, and other strange low-level things in Python. This doesn't mean I'm getting the best "bang for the buck" when it comes to performance or overhead; I'm trading these things for the ease of prototyping, debugging, and the ability to utilize other high-level interfaces. My FUSE driver won't be as performant as it would have been had I written it in C. My server monitoring suite consumes more RAM than an equivalent solution in C. When it comes down to it, if you want it done efficiently, use C. ;) (As an aside, you -can- create hideous frankenstein monsters by using compiled CPython modules with glue code between the driver API, e.g. FUSE, and your CPython driver implementation; but in that case you're adding the overhead of Python for no gain whatsoever.) For anything system critical, Python might not be the best choice. Malware removal tools are themselves the target of malware (e.g. virii attempting to disable scanners and removal tools), and utilizing Python adds (IMHO) too many points of failure. Also, file fragmentation is a non-issue on all modern filesystems (ext3/4, reiser, ntfs, hfs+, etc.) as they perform live-system defragmentation to varying degrees. I have never seen a production server of mine (utilizing reiserfs) go above 11% fragmentation (non-contiguous extant allocations), and even that resolved itself within a few hours of active use. Note that non-contiguous extants are a distinct problem, reducing file read performance substantially, thus why filesystem drivers generally handle solving this problem by themselves. The other forms of fragmentation (free space fragmentation and file scattering / related-file fragmentation) substantially less so. Certain filesystems have features designed to avoid the latter (e.g. squashfs for ordering files on bootable CDs) and the former becomes more of an issue as you attempt to allocate extremely large contiguous files. (Becoming worse as free space is exhausted as more and more files of greater size need to be shuffled around the disk platter in order to free up a contiguous run of extants.) Hope this helps, - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: How suitable is Python to write system utilities?
On 2011-01-06 06:38:24 -0800, David Boddie said: Just out of interest, which module/package are you using to examine ELF files? http://pypi.python.org/pypi/elffile - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Streaming templating languages for use as WSGI body.
On 2011-01-06 10:00:39 -0800, Adam Tauno Williams said: With HTTP/1.0 [and WSGI is HTTP/1.0 only] you have to provide a Content-Length header - so you have to generate the entire response at once [however you want to muddy "at once"]. Both of these statements are false. Streaming responses to the client requires Chunked-Encoding [HTTP/1.1] which is not possible via WSGI. This is also false. Oh for three, please try again. :) - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Can I have both Python 2.7 and Python 3.1 at the same time on the Mac?
On 2011-01-06 11:44:13 -0800, Bill Felton said: I've also seen various resources indicate that one can install both Python 2.7 and Python 3.1 -- but when I did this, I get no end of problems in the 2.7 install. I have Apple's native Python installations (2.5 and 2.6, I believe), plus Python 2.7 and 3.1 from the Python.org site (.pkg installers) installed simultaneously without a problem, but I don't use Python GUI toolkits on my Mac. Still waiting on a .pkg installer for 3.2b2, though. :( - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Streaming templating languages for use as WSGI body.
On 2011-01-06 11:11:27 -0800, Adam Tauno Williams said: On Thu, 2011-01-06 at 11:07 -0800, Alice Bevan–McGregor wrote: On 2011-01-06 10:00:39 -0800, Adam Tauno Williams said: With HTTP/1.0 [and WSGI is HTTP/1.0 only] you have to provide a Content-Length header - so you have to generate the entire response at once [however you want to muddy "at once"]. Both of these statements are false. Both these statements are true! I suggest you consult the HTTP spec. It's generally polite to provide direct references, either sections or actual links when asking someone to RTFM. No matter, examining the HTTP/1.0 RFC (conveniently chopped up and HTML-ified by the w3) I find evidence to support your argument: http://www.w3.org/Protocols/HTTP/1.0/draft-ietf-http-spec.html#Entity-Body However, HTTP clients are smarter than the raw spec. ;) Run the code found at the following link and poke the wsgiref server that is run in a web browser, with curl, or any other HTTP tool, even telnet: http://pastie.textmate.org/1435415 You'll notice no content-length header (wsgiref adds one automatically for single-element iterables) and no difficulty in receiving the entire response body, even without a content-length. The de-facto standard behaviour combined with the following text from WSGI makes streaming content with non-deterministic lengths completely reasonable: WSGI servers, gateways, and middleware must not delay the transmission of any block; they must either fully transmit the block to the client, or guarantee that they will continue transmission even while the application is producing its next block. Point me to a HTTP client from the last 10 years that doesn't handle this particular condition and I'll believe your original statements. :) - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Streaming templating languages for use as WSGI body.
On 2011-01-07 07:17:33 -0800, Michael Ströder said: As I read section 7.2.2 (Length) the Content-length header is only required in HTTP *requests* if the body contains data. According to the text it's not required in HTTP *responses*. You are correct; I mis-read that section in my haste. - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: apscheduler error
Howdy! On 2011-01-07 17:08:28 -0800, linna li said: I tried to use the apscheduler and used the sample code below from the tutorial, but got the error message: Exception in thread APScheduler (most likely raised during interpreter shutdown). What's going on here? I really appreciate any help! After talking a bit with Alex Grönholm it seems this is an issue raised fairly often (not always in the context of this package) and is not really a problem with APScheduler. It has far more to do with attempting to start a thread, then immediately exiting the main thread. That's not how threading is supposed to be used, so don't do it. ;) APScheduler 2.0 adds some improved examples, according to Alex, that don't suffer the "problem" demonstrated by the short code snippit you provided. A package of mine, TurboMail, suffers from the same threading issue if used improperly; you enqueue e-mail, it starts a thread, then you immediately exit. TM tries to work around the issue, but in most cases that workaround does not work properly. (You get strange uncatchable exceptions printed on stderr though AFIK the e-mail does get sent correctly, your application may hang waiting for the thread pool to drain if you have a "minimum thread count" option set.) I hope this clears things up a bit, - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python use growing fast
On 2011-01-10 13:02:09 -0800, MRAB said: On 10/01/2011 20:29, Dan Stromberg wrote: ...despite our wikipedia page whose first paragraph almost seems like it was written with the intention of scaring off new converts, with its "unusual" comment... Indentation as a syntatitical structure is not actually unusual in any way as was recently discussed in another thread (having difficulty finding it). It shows an example of Python code, which happens to have 2 syntax errors! Wikipedia is a Wiki; everyone is free to contribute and correct mistakes. - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: apscheduler error
On 2011-01-10 17:23:34 -0800, linna li said: I see the latest version is APScheduler 1.3.1. Where can I get APScheduler 2.0? https://bitbucket.org/agronholm/apscheduler/ I don't think 2.0 has been released yet, but that is the version number in apscheduler/__init__.py on HG tip. The examples, BTW, just add time.sleep() calls. ;) - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Ideas for a module to process command line arguments
On 2011-01-10 21:18:41 -0800, Sohail said: Hey, every body has their own favorite method/ways to process command line arguments. I've worked on a little CPython extension to handle command line arguments may be you'll find it interesting and useful Even I've implemented my own way to handle command-line scripts, marrow.script: https://github.com/pulp/marrow.script The idea with mine that you write a Python function... and that's it. The latest version has experimental support for class-based "subcommand" dispatch, but it needs work, needs to be updated to support sub-sub commands, and the help text generator needs to be overhauled to support classes properly. The argument list, typecasting, etc. is built from the argspec. Help text is pulled from the docstring. Decorators are provided to override short names, define explicit callbacks or typecasting functions, etc. I got tired of using PasteScript and OptParse. Mostly OptParse, actually. :/ - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Ideas for a module to process command line arguments
On 2011-01-11 00:32:32 -0800, Michele Simionato said: On Jan 11, 8:25 am, Alice Bevan–McGregor wrote: I got tired of using PasteScript and OptParse. Mostly OptParse, actually. :/ It's a pity that the argument parsing modules in the standard library are so verbose that everybody is reinventing the same thing :-( It looks like you have reinvented plac: http://pypi.python.org/pypi/plac And a package called `commandline`. There are many command line parsing modules, many of which are unmaintained, few have reached 1.0. My criteria for 1.0? 100% unit test coverage, complete documentation, compatibility with 2.6+ and 3.1+ within a single package. marrow.script meets that criteria, do the others? :) - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Ideas for a module to process command line arguments
On 2011-01-11 00:32:32 -0800, Michele Simionato said:
It's a pity that the argument parsing modules in the standard library
are so verbose that everybody is reinventing the same thing :-( It
looks like you have reinvented plac: http://pypi.python.org/pypi/plac
After looking into it, Plac's default help display isn't very helpful;
you need to massage your application a fair amount before generating
nice, complete-looking argument lists and such. For example:
def main(verbose: ('prints more info', 'flag', 'v'), dsn: 'connection
string'):
@annotate(dsn="connection string", verbose="prints more info")
def main(dsn, verbose=False):
The latter is marrow.script, and even without the annotation a more
complete help text is generated. The -v and flag nature are assumed
from the first unique character and default value. (Flags, when
present on the command line, invert the default value.) Py3k
annotations haven't been implemented yet, though.
Also included is an easy way to simulte command-line execution (i.e. by
reading arguments passed by hand and by returning the exit code, vs.
reading sys.argv and calling sys.exit()) for unit testing purposes.
Plac appears (from the documentation) to be written on top of argparse.
:(
- Alice.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Ideas for a module to process command line arguments
On 2011-01-11 21:41:24 -0800, Michele Simionato said: Originally plac too was able to recognize flags automatically by looking at the default value (if the default value is a boolean then the option is a flag); however I removed that functionality because I wanted to be able to differentiate between flag and (smart) options (see http://micheles.googlecode.com/hg/plac/doc/plac.html#scripts-with-options-and-smart-options). Not entirely sure what you mean by 'smart' options. If your'e referring to using a single hyphen and a list of characters to represent a long option (which, to the rest of the world, use two leading hyphens) then that's pretty weird. ;) Consider most of the GNU tools: ls -lvh tar -xzvf file.tgz (goes so far as to make the leading hyphen optional!) less -ceF logfile bc -qw ps -aux (same as tar) And even third-party tools: mysql -fH pg_dump -abO ... One major system in the world that doesn't really differentiate between long and short options is... DOS, and by extension, Windows. But they also use / as a switch character. Anyway; I'm happy with what I have wrought (and am continuing to update with support for class-based sub-command dispatch) and will be utilizing it for all scripts in the Marrow suite. To each their own, but reinvention itself can be for motivations other than NIH. I wanted something pure-Python, portable across the 3k barrier without code modification (no 2to3), that didn't use optparse, getopt, or argparse and basically be a translation layer. It can be simpler than that, as marrow.script demonstrates. - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python use growing fast
On 2011-01-10 19:49:47 -0800, Roy Smith said: One of the surprising (to me, anyway) uses of JavaScript is as the scripting language for MongoDB (http://www.mongodb.org/). I just wish they'd drop spidermonkey and go with V8 or another, faster and more modern engine. :( - Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: do you know what's CGI? (web history personal story)
On 2011-01-15 08:15:25 -0800, Sherm Pendley said: "Captain Obvious" writes: XL> ... i recall, i stopped doing Mathematica in 1998 because it's a XL> career dead-end as a programing lang, and dived into the utterly XL> idiotic Perl & unix & mysql world. (See: The Unix Pestilence ◇ Xah XL> Lee's Computing Experience (Impression Of Lisp from Mathematica).) I guess you're calling "idiotic" everything you're too lazy to understand. That's Xah for you. It's a bad sign when people use your name as a running joke in multiple mailing lists (outside the ones regularly posted in, specificaly I noticed this in one of the web framework lists) to mean "silly/stupid comments with no basis in reality". — Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: do you know what's CGI? (web history personal story)
On 2011-01-15 08:55:47 -0800, rantingrick said: On Jan 15, 10:38 am, Grant Edwards wrote: Yeah, James Cameron made a *ton* of money using it to make Avatar. Too bad he couldn't have used it to make a better movie. I found the graphics impressive; the "blue people" was merely an effort to avoid a more clear representation of the Na'vi as North American indigenous people. Avatar was very disappointing (Both in graphics and story) but maybe i expect too much...? The story was clearly "Pocahontas… in Space!", which was very disappointing. I found the look and feel of Beowulf to be more lifelike. That's just the naked Angelina Jolie in your brain talking. ;) — Alice. -- http://mail.python.org/mailman/listinfo/python-list
Re: __pycache__, one more good reason to stck with Python 2?
find . -name \*.pyc -exec rm -f {} \;
vs.
rm -rf __pycache__
I do not see how this is more difficult, but I may be missing something.
— Alice.
--
http://mail.python.org/mailman/listinfo/python-list
Re: __pycache__, one more good reason to stck with Python 2?
On 2011-01-19 13:01:04 -0800, Steven D'Aprano said:
I know I've seen problems executing .pyc files from the shell in the
past... perhaps I was conflating details of something else. Ah, I know!
[steve@sylar ~]$ chmod u+x toto.pyc
[steve@sylar ~]$ ./toto.pyc
: command not found ��
./toto.pyc: line 2: syntax error near unexpected token `('
./toto.pyc: line 2: `P7Mc@s dGHdS(tfooNs./
toto.pys'
... don't do that. I do not know why that would be expected to work,
ever. (Unless you're crafty and wrap a real shell script around the
.pyc, in which case it's no longer a .pyc.)
— Alice.
--
http://mail.python.org/mailman/listinfo/python-list
Re: how to read the last line of a huge file???
On 2011-01-26 02:59:26 -0800, Xavier Heruacles said:
I have do some log processing which is usually huge. The length of each
line is variable. How can I get the last line?? Don't tell me to use
readlines or something like linecache...
This is not optimum or efficient, but it works! If you want to see
what's going on, use 4 instead of 4096 and 8 instead of 8192 and add a
print statement to the bottom of the while loop. :)
import os
with open('biglogfile.log', 'r') as fh:
fh.seek(-4096, os.SEEK_END)
buffer = fh.read(4096)
# We are expecting a trailing newline.
while "\n" not in buffer[:-1]:
fh.seek(-8192, os.SEEK_CUR)
buffer = fh.read(4096) + buffer
# Eliminate empty lines, they aren't useful.
lines = [line for line in buffer.split('\n') if line]
print lines[-1]
— Alice. :)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Turbogears 2.1 with mako tmplates
On 2011-02-07 07:13:43 -0800, Vineet Deodhar said: For web-based solutions, I have started learning TG 2.1 By and large, the documentation on TG 2.1 official site is a work-in-process. As regards to the templates, it tells how to go about Genshi. I wish to go with mako. Gone through the docs of mako (they are good). But still, it would be better if I can get a tutorial explaining using TG 2.1 with mako. (It will save me from re-inventing the wheel, if somebody has written docs on these lines). Can anybody point to any such tutorial? (I googled, but couldn't locate any convincing result). I believe you will have signifigantly greater success having TurboGears questions answered when asked on the appropriate forum: http://groups.google.com/group/turbogears — Alice. -- http://mail.python.org/mailman/listinfo/python-list
