Re: multithreading
On 4/8/2012 7:04, Bryan wrote: Kiuhnm wrote: My question is this: can I use 'threading' without interfering with the program which will import my module? Yes. The things to avoid are described at the bottom of: http://docs.python.org/library/threading.html On platforms without threads, 'import threading' will fail. There's a standard library module dummy_threading which offers fake versions of the facilities in threading. It suggests: try: import threading as _threading except ImportError: import dummy_threading as _threading I have a decorator which takes an optional argument that tells me whether I should use locks. Maybe I could import 'threading' only if needed. If the user wants to use locks I'll assume that 'threading' is available on his/her system. By the way, should I avoid to 'import threading' more than once? Thank you so much for answering my question. Kiuhnm -- http://mail.python.org/mailman/listinfo/python-list
f python?
hi guys,
sorry am feeling a bit prolifit lately.
today's show, is: 〈Fuck Python〉
http://xahlee.org/comp/fuck_python.html
Fuck Python
By Xah Lee, 2012-04-08
fuck Python.
just fucking spend 2 hours and still going.
here's the short story.
so recently i switched to a Windows version of python. Now, Windows
version takes path using win backslash, instead of cygwin slash. This
fucking broke my find/replace scripts that takes a dir level as input.
Because i was counting slashes.
Ok no problem. My sloppiness. After all, my implementation wasn't
portable. So, let's fix it. After a while, discovered there's the
「os.sep」. Ok, replace 「"/"」 to 「os.sep」, done. Then, bang, all hell
went lose. Because, the backslash is used as escape in string, so any
regex that manipulate path got fucked majorly. So, now you need to
find a quoting mechanism. Then, fuck python doc incomprehensible
scattered comp-sci-r-us BNF shit. Then, fuck python for “os.path” and
“os” modules then string object and string functions inconsistent
ball. And FUCK Guido who wants to fuck change python for his idiotic
OOP concept of “elegance” so that some of these are deprecated.
So after several exploration of “repr()”, “format()”, “‹str›.count()”,
“os.path.normpath()”, “re.split()”, “len(re.search().group())” etc,
after a long time, let's use “re.escape()”. 2 hours has passed. Also,
discovered that “os.path.walk” is now deprecated, and one is supposed
to use the sparkling “os.walk”. In the process of refreshing my
python, the “os.path.walk” semantics is really one fucked up fuck.
Meanwhile, the “os.walk” went into incomprehensible OOP object and
iterators fuck.
now, it's close to 3 hours. This fix is supposed to be done in 10 min.
I'd have done it in elisp in just 10 minutes if not for my
waywardness.
This is Before
def process_file(dummy, current_dir, file_list):
current_dir_level = len(re.split("/", current_dir)) -
len(re.split("/", input_dir))
cur_file_level = current_dir_level+1
if min_level <= cur_file_level <= max_level:
for a_file in file_list:
if re.search(r"\.html$", a_file, re.U) and
os.path.isfile(current_dir + "/" + a_file):
replace_string_in_file(current_dir + "/" + a_file)
This is After
def process_file(dummy, current_dir, file_list):
current_dir = os.path.normpath(current_dir)
cur_dir_level = re.sub( "^" + re.escape(input_dir), "",
current_dir).count( os.sep)
cur_file_level = cur_dir_level + 1
if min_level <= cur_file_level <= max_level:
for a_file in file_list:
if re.search(r"\.html$", a_file, re.U) and
os.path.isfile(current_dir + re.escape(os.sep) + a_file):
replace_string_in_file(current_dir + os.sep + a_file)
# print "%d %s" % (cur_file_level, (current_dir + os.sep +
a_file))
Complete File
# -*- coding: utf-8 -*-
# Python
# find & replace strings in a dir
import os, sys, shutil, re
# if this this is not empty, then only these files will be processed
my_files = []
input_dir = "c:/Users/h3/web/xahlee_org/lojban/hrefgram2/"
input_dir = "/cygdrive/c/Users/h3/web/zz"
input_dir = "c:/Users/h3/web/xahlee_org/"
min_level = 2; # files and dirs inside input_dir are level 1.
max_level = 2; # inclusive
print_no_change = False
find_replace_list = [
(
u"""http://xahlee.org/
footer.html">""",
u"",
),
]
def replace_string_in_file(file_path):
"Replaces all findStr by repStr in file file_path"
temp_fname = file_path + "~lc~"
backup_fname = file_path + "~bk~"
# print "reading:", file_path
input_file = open(file_path, "rb")
file_content = unicode(input_file.read(), "utf-8")
input_file.close()
num_replaced = 0
for a_pair in find_replace_list:
num_replaced += file_content.count(a_pair[0])
output_text = file_content.replace(a_pair[0], a_pair[1])
file_content = output_text
if num_replaced > 0:
print "◆ ", num_replaced, " ", file_path.replace("\\", "/")
shutil.copy2(file_path, backup_fname)
output_file = open(file_path, "r+b")
output_file.read() # we do this way instead of “os.rename” to
preserve file creation date
output_file.seek(0)
output_file.write(output_text.encode("utf-8"))
output_file.truncate()
output_file.close()
else:
if print_no_change == True:
print "no change:", file_path
# os.remove(file_path)
# os.rename(temp_fname, file_path)
def process_file(dummy, current_dir, file_list):
current_dir = os.path.normpath(current_dir)
cur_dir_level = re.sub( "^" + re.escape(input_dir), "",
current_dir).count( os.sep)
cur_file_level = cur_dir_level + 1
if min_level <= cur_file_level <= max_level:
for a_file in file_list:
if re.search(r"\.html$", a_file, re.U) and
os.path.isfile(current_dir + re.escape(os.sep) + a_file):
replace_string_in_file(current_dir + os.sep + a_file)
# print "%d %s" % (cur_file_level, (current_dir + os.sep
Re: f python?
On Sun, 08 Apr 2012 04:11:20 -0700, Xah Lee wrote:
[...]
I have read Xah Lee's post so that you don't have to.
Shorter Xah Lee:
"I don't know Python very well, and rather than admit I made
some pretty lousy design choices in my code, I blame Python.
And then I cross-post about it, because I'm the most important
person in the Universe."
When the only tool you know how to use is a hammer, everything looks like
a nail. Instead of using regexes ("now you have two problems"), use the
right tool: to count path components, split the path, then count the
number of path components directly.
import os
components = os.path.split(some_path)
print len(components)
No matter what separator the OS users, os.path.split will do the right
thing. There's no need to mess about escaping separators so you can
hammer it with the regex module, when Python comes with the perfectly
functional socket-wrench you actually need.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: f python?
Congratulations. You just tried to treat Python as though it were a hammer and your task as though it were a nail. And you bashed your thumb while doing it. You also hurt yourself on Windows and blamed it on Python. I'm afraid I'm fresh out of sympathy, won't get a new shipment till tomorrow. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On Apr 8, 4:34 am, Steven D'Aprano wrote:
> On Sun, 08 Apr 2012 04:11:20 -0700, Xah Lee wrote:
>
> [...]
>
> I have read Xah Lee's post so that you don't have to.
>
> Shorter Xah Lee:
>
> "I don't know Python very well, and rather than admit I made
> some pretty lousy design choices in my code, I blame Python.
> And then I cross-post about it, because I'm the most important
> person in the Universe."
>
> When the only tool you know how to use is a hammer, everything looks like
> a nail. Instead of using regexes ("now you have two problems"), use the
> right tool: to count path components, split the path, then count the
> number of path components directly.
>
> import os
> components = os.path.split(some_path)
> print len(components)
>
> No matter what separator the OS users, os.path.split will do the right
> thing. There's no need to mess about escaping separators so you can
> hammer it with the regex module, when Python comes with the perfectly
> functional socket-wrench you actually need.
Lol. i think you tried to make fun of me too fast.
check your code.
O, was it you who made fun of my python tutorial before? i was busy,
i'll have to get back on that down the road.
Xah
--
http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On Sun, 08 Apr 2012 11:34:56 +, Steven D'Aprano wrote: > I have read Xah Lee's post so that you don't have to. Well, I certainly shall not be reading - or even seeing - any more of his drivel. -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On 08/04/2012 12:11, Xah Lee wrote: Hi Xah, You clearly didn't want help on this subject, as you really now how to do it anyway. But having read your posts over the years, I'd like to give you an observation on your persona, free of charge! :-) You are actually a talented writer, some may find your occasional profanity offensive but at least it highlights your frustration. You are undoubtedly and proven a good mathematian and more important than that self taught. You have a natural feel for design (otherwise you would not clash with others view of programming). You know a mixture of programming languages. Whether you like it or not, you are in the perfect position to create a new programming language and design a new programming paradigm. Unhindered from all the legacy crap, that keep people like me behind (I actually like BNF for example). It is likely I am wrong, but if that is your destiny there is no point fighting it. Cheers and good luck, Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On Sun, 08 Apr 2012 11:34:56 +, Steven D'Aprano wrote:
> When the only tool you know how to use is a hammer, everything looks
> like a nail. Instead of using regexes ("now you have two problems"), use
> the right tool: to count path components, split the path, then count the
> number of path components directly.
>
> import os
> components = os.path.split(some_path)
> print len(components)
Which is completely wrong. How embarrassing. Serves me right for not
testing my code before sending.
Nevertheless it is easy enough to write a split path function, or even to
use it in place:
def splitpath(path):
return os.path.normpath(path).split(os.path.sep)
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python Gotcha's?
In article , John Nagle wrote: > 1. Nobody is really in charge of third party packages. In the > Perl world, there's a central repository, CPAN, and quality > control. Python's "pypi" is just a collection of links. Many > major packages are maintained by one person, and if they lose > interest, the package dies. While I agree that this is a problem, it's not specifically a Python problem. There's a lot of abandonware out there. In all languages. With both OSS and commercial products. Having an official curated central repository is a good thing, but it has its down side too. What happens when the curator decides not to allow your code into the library? Down that path lies things like the Apple Store for IOS. If Apple decides they don't want your app for whatever reason, your app is dead. -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On 04/08/12 09:11, HoneyMonster wrote: Well, I certainly shall not be reading - or even seeing - any more of his drivel. Yes you will. There is always someone willing to include his entire messages for a one line reply. It's always September somewhere. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/| and a sheep voting on +1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner. IM: da...@vex.net -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
Xah Lee wrote: >hi guys, > >sorry am feeling a bit prolifit lately. > >today's show, is: 'Fuck Python' >http://xahlee.org/comp/fuck_python.html > > >Fuck Python > By Xah Lee, 2012-04-08 > >fuck Python. > >just fucking spend 2 hours and still going. > >here's the short story. > >so recently i switched to a Windows version of python. Now, Windows >version takes path using win backslash, instead of cygwin slash. This >fucking broke my find/replace scripts that takes a dir level as input. >Because i was counting slashes. > >Ok no problem. My sloppiness. After all, my implementation wasn't >portable. So, let's fix it. After a while, discovered there's the >'os.sep'. Ok, replace "/" to 'os.sep', done. Then, bang, all hell >went lose. Because, the backslash is used as escape in string, so any >regex that manipulate path got fucked majorly. When Microsoft created MS-DOS, they decided to use '\' as the separator in file names. This was at a time when several previously existing interactive operating systems were using '/' as the file name separator and at least one was using '\' as an escape character. As a result of Microsoft's decision to use '\' as the separator, people have had to do extra work to adapt programs written for Windows to run in non-Windows environments, and vice versa. People have had to do extra work to write software that is portable between these environments. People have done extra work while creating tools to make writing portable software easier. And people have to do extra work when they use these tools, because using them is still harder than writing portable code for operating systems that all used '/' as their separator would have been. If you added up the cost of all the extra work that people have done as a result of Microsoft's decision to use '\' as the file name separator, it would probably be enough money to launch the Burj Khalifa into geosynchronous orbit. So, when you say fuck Python, are you sure you're shooting at the right target? -- David Canzi | TIMTOWWTDI (tim-toe-woe-dee): There Is More Than One | Wrong Way To Do It -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
["Followup-To:" header set to comp.lang.lisp.] On 2012-04-08, David Canzi wrote: > Xah Lee wrote: >>hi guys, >> >>sorry am feeling a bit prolifit lately. >> >>today's show, is: 'Fuck Python' >>http://xahlee.org/comp/fuck_python.html >> >> >>Fuck Python >> By Xah Lee, 2012-04-08 >> >>fuck Python. >> >>just fucking spend 2 hours and still going. >> >>here's the short story. >> >>so recently i switched to a Windows version of python. Now, Windows >>version takes path using win backslash, instead of cygwin slash. This >>fucking broke my find/replace scripts that takes a dir level as input. >>Because i was counting slashes. >> >>Ok no problem. My sloppiness. After all, my implementation wasn't >>portable. So, let's fix it. After a while, discovered there's the >>'os.sep'. Ok, replace "/" to 'os.sep', done. Then, bang, all hell >>went lose. Because, the backslash is used as escape in string, so any >>regex that manipulate path got fucked majorly. > > When Microsoft created MS-DOS, they decided to use '\' as > the separator in file names. This is false. The MS-DOS (dare I say it) "kernel" accepts both forward and backslashes as separators. The application-level choice was once configurable through a variable in COMMAND.COM. Then they hard-coded it to backslash. However, Microsoft operating systems continued to (and until this day) recognize slash as a path separator. Only, there are broken userland programs on Windows which don't know this. > So, when you say fuck Python, are you sure you're shooting at the > right target? I would have to say, probably yes. -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On 2012-04-08 17:03, David Canzi wrote: > If you added up the cost of all the extra work that people have > done as a result of Microsoft's decision to use '\' as the file > name separator, it would probably be enough money to launch the > Burj Khalifa into geosynchronous orbit. So we have another contender for the Most Expensive One-byte Mistake? Poul-Henning Kamp nominated the C/Unix guys: http://queue.acm.org/detail.cfm?id=2010365 hp -- _ | Peter J. Holzer| Deprecating human carelessness and |_|_) | Sysadmin WSR | ignorance has no successful track record. | | | h...@hjp.at | __/ | http://www.hjp.at/ | -- Bill Code on a...@irtf.org -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
"David Canzi" wrote: >Xah Lee wrote: Please check whom you are replying to. Do not feed the trolls, please. jue -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Gotcha's?
> 8. Opening a URL can result in an unexpected prompt on > standard input if the URL has authentication. This can > stall servers. Can you give an example? I don't think anything in the standard library does that. -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On 4/8/2012 7:11 AM, Xah Lee wrote:
so recently i switched to a Windows version of python. Now, Windows
version takes path using win backslash, instead of cygwin slash.
Python passes strings to Windows functions as the user requests. In
spite of the fact that command.com and cmd.exe (command prompt window)
require backslashes for paths that they proccess, the internal Windows
functions all (as far as I know) accept slashes in paths. Slashes are
used for options only at the shell level. So within a Python program,
one can use slashes on Windows too, just as you are used to.
open("C:/users/terry/data/somefile.txt")
I just verified that you can pass arguments with slashes, such as a
path, to a Python program from the shell. Here is my tem.py:
import sys
print(sys.argv)
If I run it from its directory as current path:
F:\Python\mypy>c:\programs\python32\python.exe tem.py /a/b
['tem.py', '/a/b']
If the current directory were not the directory containing the script,
then one would have to use backslashes for the path to the script and
would see backslashes in sys.argv[0]. If I execute tem.py in IDLE, the
output is
['F:\\Python\\mypy\\tem.py']
because IDLE 'starts in' the python.exe directory and Windows functions
produce paths with '\' even though they accept them with '/'.
Try aiming your ire at the right target. The title of this post should
have been Fuck Microsoft for the billion (at least) in time, effort, and
annoyance they inflicted on the world by being intentionally different
from and conflicting with unix on shell syntax.
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Question on Python 3 shell restarting
How may I get a fresh Python shell with Idle 3.2 ? I have to run the same modules several times with all variables cleared. Thanks, franck -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Gotcha's?
On Mon, Apr 9, 2012 at 3:55 AM, Miki Tebeka wrote:
>> 8. Opening a URL can result in an unexpected prompt on
>> standard input if the URL has authentication. This can
>> stall servers.
> Can you give an example? I don't think anything in the standard library does
> that.
I just threw together a quick test with a page on my local server that
returns 401 Unauth and calls for basic authentication. With Python 2:
Python 2.4.5 (#1, Jul 22 2011, 02:01:04)
[GCC 4.1.1] on mingw32
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib
>>> urllib.urlopen("http://CENSORED";)
Enter username for CENSORED at CENSORED:
Warning: Problem with getpass. Passwords may be echoed.
Enter password for in CENSORED at CENSORED:
>
I hit enter twice and that's what it did. (The "Enter password for
in" presumably would be quoting back my username.)
Using the same URL with urllib2.urlopen() threw an HTTPError exception
citing the 401.
In Python 3, urllib.request.urlopen() throws HTTPError as above.
So it seems that this gotcha, while a very real problem, is only an
issue with a deprecated module on the old branch of Python. Unless
it's somewhere else that I couldn't find?
ChrisA
--
http://mail.python.org/mailman/listinfo/python-list
Interrupting a blocking function frolm another thread.
I am using the python module nfqueue-bindings which is a nfqueue packet intercepting module. It uses the following snippet of code to start the process: print "trying to run" try: q.try_run() except KeyboardInterrupt, e: print "interrupted" The q.try_run() method blocks. I would like to be able to interrupt this the same way you can hit control-c to unblock it, but from another thread. Does anyone have any idea on how to do this? Is there some sort of Exception I can generate from another thread that would do this? Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Gotcha's?
On 4/8/2012 10:55 AM, Miki Tebeka wrote: 8. Opening a URL can result in an unexpected prompt on standard input if the URL has authentication. This can stall servers. Can you give an example? I don't think anything in the standard library does that. It's in "urllib". See http://docs.python.org/library/urllib.html "When performing basic authentication, a FancyURLopener instance calls its prompt_user_passwd() method. The default implementation asks the users for the required information on the controlling terminal. A subclass may override this method to support more appropriate behavior if needed." A related "gotcha" is knowing that "urllib" sucks and you should use "urllib2". John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On 2012-04-08, Peter J. Holzer wrote:
> On 2012-04-08 17:03, David Canzi wrote:
>> If you added up the cost of all the extra work that people have
>> done as a result of Microsoft's decision to use '\' as the file
>> name separator, it would probably be enough money to launch the
>> Burj Khalifa into geosynchronous orbit.
>
> So we have another contender for the Most Expensive One-byte Mistake?
The one byte mistake in DOS and Windows is recognizing two characters as path
separators. All code that correctly handles paths is complicated by having to
look for a set of characters instead of just scanning for a byte.
> http://queue.acm.org/detail.cfm?id=2010365
DOS backslashes are already mentioned in that page, but alas it perpetuates the
clueless myth that DOS and windows do not recognize any other path separator.
Worse, the one byte Unix mistake being covered is, disappointingly, just a
clueless rant against null-terminated strings.
Null-terminated strings are infinitely better than the ridiculous encapsulation
of length + data.
For one thing, if s is a non-empty null terminated string then, cdr(s) is also
a string representing the rest of that string without the first character,
where cdr(s) is conveniently defined as s + 1.
Not only can compilers compress storage by recognizing that string literals are
the suffixes of other string literals, but a lot of string manipulation code is
simplified, because you can treat a pointer to interior of any string as a
string.
Because they are recursively defined, you can do elegant tail recursion on null
terminated strings:
const char *rec_strchr(const char *in, int ch)
{
if (*in == 0)
return 0;
else if (*in == ch)
return in;
else
return rec_strchr(in + 1, ch);
}
length + data also raises the question: what type is the length field? One
byte? Two bytes? Four? And then you have issues of byte order. Null terminated
C strings can be written straight to a binary file or network socket and be
instantly understood on the other end.
Null terminated strings have simplified all kids of text manipulation, lexical
scanning, and data storage/communication code resulting in immeasurable
savings over the years.
--
http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On Mon, Apr 9, 2012 at 5:14 AM, Kaz Kylheku wrote: > Not only can compilers compress storage by recognizing that string literals > are > the suffixes of other string literals, but a lot of string manipulation code > is > simplified, because you can treat a pointer to interior of any string as a > string. I'm not sure about the value of tail recursion in C, but this is definitely a majorly useful feature, as is the related technique of parsing by dropping null bytes into the string (see for instance the strtok function, which need not do any memory movement; I wrote a CSV parser that works the same way). Often I use both techniques simultaneously, for instance in parsing this sort of string: "A:100 B:200 C:300" First, tokenize on the spaces by looking for a space, retaining a pointer, and putting in a NUL: char *next=strchr(str,' '); if (!next) break; *next++=0; Then read a character, and increment the pointer through that string as you parse. Try doing THAT in a high level language without any memory copying. And "without any memory copying" may not be important with this trivial example, but suppose you've just read in a huge CSV file to parse - maybe 16MB in the normal case, with no actual limit other than virtual memory. (And yes, I read the whole thing in at once, because it comes from a Postgres database and reading it in pieces would put more load on the central database server.) Don't get me wrong, I wouldn't want to do _everything_ in C; but I also wouldn't want to do everything in length-preceded strings. The nearest equivalent that would be able to use the shared buffer is a length-external string like BASIC uses (or used, back when I used to write BASIC code and 8086 assembly to interface with it) - a string "object" consists of a length and a pointer. But that has issues with freeing up memory, if you're using parts of a string. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: GUIs - a modest proposal
On Mar 2, 6:42 am, lkcl wrote: > ah. right. you're either referring to pyjampiler (in the pyjs > world) or to > [...] > the former actually got taken to an extreme by a group who embedded > the pyjs 0.5 compiler into their application environment, i keep > forgetting > what it's called. http://nagare.org. took me a while to remember. sorry about that. l. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Gotcha's?
* Grzegorz Staniak wrote: > On 06.04.2012, rusi wroted: > >> There should be one-- and preferably only one --obvious way to do it. > > Then again, practicality beats purity. Yes. If you ever grepped for, say, the usage of dictionary keys in a bigger application, you might agree, that having multiple equivalent quote characters is not as practical as it might seem in the first place. Code is written once and read often. nd -- http://mail.python.org/mailman/listinfo/python-list
Re: Number of languages known [was Re: Python is readable] - somewhat OT
On 03/30/2012 06:25 AM, Steve Howell wrote: On Mar 29, 11:53 am, Devin Jeanpierre wrote: Well, what sort of language differences make for English vs Mandarin? Relational algebraic-style programming is useful, but definitely a large language barrier to people that don't know any SQL. I think this is reasonable. (It would not matter even if you gave SQL python-like syntax, the mode of thinking is different, and for a good reason.) I don't see any fundamental disconnect between SQL thinking and Python thinking. List comprehensions are very close to SQL SELECTs semantically, and not that far off syntactically. [row.x for row in foo if x == 3] select x from foo where x = 3 which is where most people get it wrong; the SQL SELECTs statement did not specify how the machine is going to get its answer, while the Python's list comprehension explicitly specify that the machine is going to loop over foo. In most implementation of SQL with the proper indexes set up, the SELECT statement above will most likely just use its index to avoid looping over the whole foo, and the most smartest ones might notice that the result query only ever contains 3 and so just use the count of the index (I don't know if any existing SQL engine is *that* smart though). -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing & Logging
Le 08/04/2012 02:56, Vinay Sajip a écrit : Thibaut gmail.com> writes: This is exactly what I wanted, it seems perfect. However I still have a question, from what I understood, I have to configure logging AFTER creating the process, to avoid children process to inherits the logging config. Unless there is a way to "clean" logging configuration in children processes, so they only have one handler : the QueueHandler. I looked at the logging code and it doesn't seems to have an easy way to do this. The problem of configuring the logging after the process creation is that... I can't log during process creation. But if it's too complicated, I will just do this. I've updated the 3.2 / 3.3 logging cookbook with an example of what I mean. There is a gist of the example script at https://gist.github.com/2331314/ and the cookbook example should show once the docs get built on docs.python.org. Regards, Vinay Sajip Thank you very much, the gist example is perfect. This logging system is just perfect ! Thanks for your work and explanations. Tips about windows / posix differences are nice too. -- Thibaut -- http://mail.python.org/mailman/listinfo/python-list
Re: Interrupting a blocking function frolm another thread.
On Apr 8, 2:45 pm, "superhac...@gmail.com" wrote: > I am using the python module nfqueue-bindings which is a nfqueue > packet intercepting module. It uses the following snippet of code to > start the process: > > print "trying to run" > try: > q.try_run() > except KeyboardInterrupt, e: > print "interrupted" > > The q.try_run() method blocks. I would like to be able to interrupt > this the same way you can hit control-c to unblock it, but from > another thread. Does anyone have any idea on how to do this? Is > there some sort of Exception I can generate from another thread that > would do this? The simplest and most reliable way will be to modify the asynchronous I/O example at https://www.wzdftpd.net/redmine/projects/nfqueue-bindings/repository/entry/examples/nfq_asyncore.py to do what you want. The classical way to accomplish this would be to create a second I/O descriptor (via os.pipe or similiar) and wait for that descriptor in the same async I/O loop. When you want to stop the loop, send a message (could be a single byte) to the second descriptor and then respond appropriately in your asynchronous I/O handler. However, simply ignoring the nfqueue socket while doing other processing might be acceptable too. I would read the documentation and ask questions carefully before taking that approach, as it may lead to lost data or other problems. The viability of this approach depends heavily on your application. Adam -- http://mail.python.org/mailman/listinfo/python-list
Re: Interrupting a blocking function frolm another thread.
On Sunday, April 8, 2012 3:55:41 PM UTC-5, Adam Skutt wrote: > On Apr 8, 2:45 pm, "superhac...@gmail.com" > wrote: > > I am using the python module nfqueue-bindings which is a nfqueue > > packet intercepting module. It uses the following snippet of code to > > start the process: > > > > print "trying to run" > > try: > > q.try_run() > > except KeyboardInterrupt, e: > > print "interrupted" > > > > The q.try_run() method blocks. I would like to be able to interrupt > > this the same way you can hit control-c to unblock it, but from > > another thread. Does anyone have any idea on how to do this? Is > > there some sort of Exception I can generate from another thread that > > would do this? > > The simplest and most reliable way will be to modify the asynchronous > I/O example at > https://www.wzdftpd.net/redmine/projects/nfqueue-bindings/repository/entry/examples/nfq_asyncore.py > to do what you want. The classical way to accomplish this would be to > create a second I/O descriptor (via os.pipe or similiar) and wait for > that descriptor in the same async I/O loop. When you want to stop the > loop, send a message (could be a single byte) to the second descriptor > and then respond appropriately in your asynchronous I/O handler. > > However, simply ignoring the nfqueue socket while doing other > processing might be acceptable too. I would read the documentation > and ask questions carefully before taking that approach, as it may > lead to lost data or other problems. The viability of this approach > depends heavily on your application. > > Adam Thanks for the reply Adam. I am still confused with the example with the example you linked to. I have only been working with Python for a couple of weeks, but I am fluent with C. Given the example you linked to, are you saying that this is the run loop for asynchronous comm that a end user of the module could use? Or is this something with the nfqueue-bindings module that I would I have to modify. I don't recognize the the main loop in the example. I am not not worried about losing data, I just need to start and stop this from another thread. -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On Sun, Apr 8, 2012 at 11:32 AM, Peter J. Holzer wrote: > Poul-Henning Kamp nominated the C/Unix guys: > > http://queue.acm.org/detail.cfm?id=2010365 Besides what Kaz and Chris wrote, the suggestion that if they had chosen ptr+len format then we wouldn't have buffer overflows is erroneous. There's a difference between a string and a string buffer, and the metadata records the length of the string, not the size of the buffer, and so the gets function in that parallel universe would be just as broken as the one in ours. -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
"Kaz Kylheku" wrote in message news:20120408114313...@kylheku.com... Worse, the one byte Unix mistake being covered is, disappointingly, just a clueless rant against null-terminated strings. Null-terminated strings are infinitely better than the ridiculous encapsulation of length + data. For one thing, if s is a non-empty null terminated string then, cdr(s) is also a string representing the rest of that string without the first character, where cdr(s) is conveniently defined as s + 1. If strings are represented as (ptr,length), then a cdr(s) would have to return (ptr+1,length-1), or (nil,0) if s was one character. No big deal. (Note I saw your post in comp.lang.python; I don't about any implications of that for Lisp.) And if, instead, you want to represent all but the last character of the string, then it's just (ptr,length-1). (Some checking is needed around empty strings, but similar checks are needed around s+1.) In addition, if you want to represent the middle of a string, then it's also very easy: (ptr+a,b). Not only can compilers compress storage by recognizing that string literals are the suffixes of other string literals, but a lot of string manipulation code is simplified, because you can treat a pointer to interior of any string as a string. Yes, the string "bart" also contains "art", "rt" and "t". But with counted strintgs, it can also contain "bar", "ba", "b", etc There are a few advantages to counted strings too... length + data also raises the question: what type is the length field? One byte? Two bytes? Four? Depends on the architecture. But 4+4 for 32-bits, and 8+8 bytes for 64-bits, I would guess, for general flex strings of any length. There are other ways of encoding a length. (For example I use one short string type of maximum M characters, but the current length N is encoded into the string, without needing any extra count byte (by fiddling about with the last couple of bytes). If you're trying to store a short string in an 8-byte field in a struct, then this will let you use all 8 bytes; a zero-terminated one, only 7.) And then you have issues of byte order. Which also affects every single value of more than one byte. Null terminated C strings can be written straight to a binary file or network socket and be instantly understood on the other end. But they can't contains nulls! Null terminated strings have simplified all kids of text manipulation, lexical scanning, and data storage/communication code resulting in immeasurable savings over the years. They both have their uses. -- Bartc -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On Sun, 08 Apr 2012 04:11:20 -0700, Xah Lee wrote: > Ok no problem. My sloppiness. After all, my implementation wasn't > portable. So, let's fix it. After a while, discovered there's the > os.sep. Ok, replace "/" to os.sep, done. Then, bang, all hell > went lose. Because, the backslash is used as escape in string, so any > regex that manipulate path got fucked majorly. So, now you need to > find a quoting mechanism. if os.altsep is not None: sep_re = '[%s%s]' % (os.sep, os.altsep) else: sep_re = '[%s]' % os.sep But really, you should be ranting about regexps rather than Python. They're convenient if you know exactly what you want to match, but a nuisance if you need to generate the expression based upon data which is only available at run-time (and re.escape() only solves one very specific problem). -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On Sun, Apr 8, 2012 at 6:55 PM, Dennis Lee Bieber wrote: > The main reason, as I recall, for the command line using \ for file > paths is that it inherited / as command OPTION prefix from CP/M; MS-DOS > being a 32-bit work-alike for CP/M in the first generation. I also thought it was because Bill Gates used a PDP-11 in high school and DEC used the / as command OPTION I found out later that Gates purchased DOS rather than wrote it, so this story may be erroneous. Nevertheless, at the time, DEC was way bigger than AT&T (in computers) and the choice really wasn't surprising. -- http://mail.python.org/mailman/listinfo/python-list
pygame.Rect question
Hello, I am new to Python and began using pygame to start some game programming. I was hoping someone could help me out with something that seems simple but is really confusing me. I am creating a rect and then using the attributes of the rect to set the size and location. I set rect.left to 30, rect.top to 30 and rect.width = 20 This works fine. However, when looking at rect.right() it shows that it is equal to 50. I suppose this is equal to 30+20. However, since the first pixel is on location 30, wouldn't the 20th pixel be on 49 (not 50)? Am I missing something here? It is really confusing me when I am doing some collision algorithms. Any help? Thanks, Scott -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On 04/08/2012 08:04 PM, David Robinow wrote: > On Sun, Apr 8, 2012 at 6:55 PM, Dennis Lee Bieber > wrote: >>The main reason, as I recall, for the command line using \ for file >> paths is that it inherited / as command OPTION prefix from CP/M; MS-DOS >> being a 32-bit work-alike for CP/M in the first generation. > I also thought it was because Bill Gates used a PDP-11 in high school > and DEC used the / as command OPTION > I found out later that Gates purchased DOS rather than wrote it, so > this story may be erroneous. Nevertheless, at the time, DEC was way > bigger than AT&T (in computers) and the choice really wasn't > surprising. CP/M was indeed the inspiration for QDOS (Quick and Dirty OS), and it shows in lots of ways, right down to the layout of the PSP at the beginning of program's memory images. There was a separate group within Microsoft doing Xenix ( ~ Unix), and the two did not cooperate. But just as important, just as it was "clear" that PC programs would never need more than 640k, it was also clear that disks would never get bigger than a few meg. Subdirectories were not part of the original picture, and were not supported till MSDOS 2.0 -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame.Rect question
On 04/08/2012 07:58 PM, Scott Siegler wrote: > Hello, > > I am new to Python and began using pygame to start some game programming. I > was hoping someone could help me out with something that seems simple but is > really confusing me. > > I am creating a rect and then using the attributes of the rect to set the > size and location. > > I set rect.left to 30, rect.top to 30 and rect.width = 20 > > This works fine. However, when looking at rect.right() it shows that it is > equal to 50. I suppose this is equal to 30+20. However, since the first > pixel is on location 30, wouldn't the 20th pixel be on 49 (not 50)? > > Am I missing something here? It is really confusing me when I am doing some > collision algorithms. > > Any help? > > Thanks, > Scott I don't know about pygame, but almost everywhere in the standard library, ranges are closed at the begin and open at the end. For example, if you have range(30, 50), there are 20 items, numbered 30 through 49. I expect the same will be true for rect.right() and rect.bottom(). -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: multithreading
Kiuhnm wrote: > I have a decorator which takes an optional argument that tells me > whether I should use locks. > Maybe I could import 'threading' only if needed. If the user wants to > use locks I'll assume that 'threading' is available on his/her system. Use of dummy_threading might be cleaner. It has things with the same names as the locks in real threading, and that you can create and call just like locks, but they don't actually do anything. > By the way, should I avoid to 'import threading' more than once? No; threading imports like any other competent module. The tricky part doesn't start until you actually use its facilities. -Bryan -- http://mail.python.org/mailman/listinfo/python-list
Re: Interrupting a blocking function frolm another thread.
On Apr 8, 5:52 pm, superhac...@gmail.com wrote: > On Sunday, April 8, 2012 3:55:41 PM UTC-5, Adam Skutt wrote: > > On Apr 8, 2:45 pm, "superhac...@gmail.com" > > wrote: > > > I am using the python module nfqueue-bindings which is a nfqueue > > > packet intercepting module. It uses the following snippet of code to > > > start the process: > > > > print "trying to run" > > > try: > > > q.try_run() > > > except KeyboardInterrupt, e: > > > print "interrupted" > > > > The q.try_run() method blocks. I would like to be able to interrupt > > > this the same way you can hit control-c to unblock it, but from > > > another thread. Does anyone have any idea on how to do this? Is > > > there some sort of Exception I can generate from another thread that > > > would do this? > > > The simplest and most reliable way will be to modify the asynchronous > > I/O example > > athttps://www.wzdftpd.net/redmine/projects/nfqueue-bindings/repository/... > > to do what you want. The classical way to accomplish this would be to > > create a second I/O descriptor (via os.pipe or similiar) and wait for > > that descriptor in the same async I/O loop. When you want to stop the > > loop, send a message (could be a single byte) to the second descriptor > > and then respond appropriately in your asynchronous I/O handler. > > > However, simply ignoring the nfqueue socket while doing other > > processing might be acceptable too. I would read the documentation > > and ask questions carefully before taking that approach, as it may > > lead to lost data or other problems. The viability of this approach > > depends heavily on your application. > > > Adam > > Thanks for the reply Adam. I am still confused with the example with the > example you linked to. I have only been working with Python for a couple of > weeks, but I am fluent with C. Given the example you linked to, are you > saying that this is the run loop for asynchronous comm that a end user of the > module could use? Or is this something with the nfqueue-bindings module that > I would I have to modify. I don't recognize the the main loop in the example. asyncore is a standard python module for handling asynchronous I/O operations (i.e., select/poll operations). The main loop in that example is handled by the 'asyncore.loop()' call, which just calls select() in a loop until all channels are closed. You're looking at the standard Python library technique for an asynchronous I/O event loop. asyncore.file_dispatcher enables use of asyncore.loop() with standard UNIX FDs. nfqueue.queue objects provide access to their underlying FDs so you can wait on them. asyncore.loop() is the main loop of the application, performing endless select/poll waits and calling 'handle_read' whenever the netfilter FD has data for reading. You can find more details about the module in the Python library docs. Since your code is Linux specific, you can always call os.select() or os.poll() if it makes you feel better, but I don't think that's going to simplify anything here. Hopefully that helps you. Adam -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Script Works Locally But Not Remotely with SSH
On Sat, Apr 7, 2012 at 5:41 PM, Dennis Lee Bieber wrote: > The WinXP machine would need to have X-client translator (something > that redirects all the Windows native graphics into X protocol and ships > it to the specified server machine). > > As I recall -- such are not cheap applications. X Servers for windows aren't expensive pieces of software anymore. XMing is quite good, and free. Cygwin also has an X server, but Cygwin always seems like too much of a hassle to me. -- Jerry -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
Xah Lee wrote: « http://xahlee.org/comp/fuck_python.html » David Canzi wrote «When Microsoft created MS-DOS, they decided to use '\' as the separator in file names.  This was at a time when several previously existing interactive operating systems were using '/' as the file name separator and at least one was using '\' as an escape character.  As a result of Microsoft's decision to use '\' as the separator, people have had to do extra work to adapt programs written for Windows to run in non-Windows environments, and vice versa.  People have had to do extra work to write software that is portable between these environments. People have done extra work while creating tools to make writing portable software easier.  And people have to do extra work when they use these tools, because using them is still harder than writing portable code for operating systems that all used '/' as their separator would have been.» namekuseijin wrote: > yes, absolutely.  But you got 2 inaccuracies there:  1) Microsoft didn't > create DOS; 2) fucking DOS was written in C, and guess what, it uses \ as > escape character.  Fucking microsoft. > > > So, when you say fuck Python, are you sure you're shooting at the > > right target? > > I agree.  Fuck winDOS and fucking microsoft. No. The choice to use backslash than slash is actually a good one. because, slash is one of the useful char, far more so than backslash. Users should be able to use that for file names. i don't know the detailed history of path separator, but if i were to blame, it's fuck unix. The entirety of unix, unix geek, unixers, unix fuckheads. Fuck unix. ãOn Unix Filename Characters Problemã http://xahlee.org/UnixResource_dir/writ/unix_filename_chars.html ãOn Unix File System's Case Sensitivityã http://xahlee.org/UnixResource_dir/_/fileCaseSens.html ãUNIX Tar Problem: File Length Truncation, Unicode Name Supportã http://xahlee.org/comp/unix_tar_problem.html ãWhat Characters Are Not Allowed in File Names?ã http://xahlee.org/mswin/allowed_chars_in_file_names.html ãUnicode Support in File Names: Windows, Mac, Emacs, Unison, Rsync, USB, Zipã http://xahlee.org/mswin/unicode_support_file_names.html ãThe Nature of the Unix Philosophyã http://xahlee.org/UnixResource_dir/writ/unix_phil.html Xah -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On Mon, Apr 9, 2012 at 3:45 PM, Xah Lee wrote: > because, slash is one of the useful char, far more so than backslash. > Users should be able to use that for file names. > Users should be able to use EVERY character for their file names. So here's a solution. Your path separator is the byte 0xFF, and your file names are UTF-8 encoded. I think this will be a real winner, and you should team up with Ranting Rick to produce a new operating system and Python with this new specification and RULE THE WORLD! Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
