argparse.ArgumentParser formatter_class argument
Hello, As far as I can see it is currently impossible to apply more than one class to an ArgumentParser. For example, I'd like to use both RawDescriptionHelpFormatter *and* ArgumentDefaultsHelpFormatter in an ArgumentParser, but it seems that's impossible, as one can only choose a single one. Any suggestions? -- Seb -- https://mail.python.org/mailman/listinfo/python-list
Re: argparse.ArgumentParser formatter_class argument
On Sun, 17 Dec 2017 10:12:07 +0100, Peter Otten <__pete...@web.de> wrote: > Seb wrote: >> As far as I can see it is currently impossible to apply more than one >> class to an ArgumentParser. For example, I'd like to use both >> RawDescriptionHelpFormatter *and* ArgumentDefaultsHelpFormatter in an >> ArgumentParser, but it seems that's impossible, as one can only >> choose a single one. Any suggestions? > Try > class MyHelpFormatter( argparse.RawDescriptionHelpFormatter, > argparse.ArgumentDefaultsHelpFormatter ): pass > parser = argparse.ArgumentParser(formatter_class=MyHelpFormatter) Yes, that worked. Thanks, -- Seb -- https://mail.python.org/mailman/listinfo/python-list
matrix multiplication
Hello, The following is an example of an Nx3 matrix (`uvw`) representing N vectors that need to be multiplied by a 3x3 matrix (generated by `randint_mat` function) and store the result in `uvw_rots`: ---<cut here---start--->--- import numpy as np def randint_mat(x): return np.random.randint(x, size=(3, 3)) np.random.seed(123) uvw = np.random.randn(1000, 3) maxint = np.random.randint(1, 10, size=uvw.shape[0]) uvw_rots = np.empty_like(uvw) for i, v in enumerate(maxint): mati = randint_mat(v) uvw_roti = np.dot(uvw[i], mati) uvw_rots[i] = uvw_roti ---<cut here---end->--- Is there a better (faster) approach than looping through the rows of uvw as shown? Thanks, -- Seb -- https://mail.python.org/mailman/listinfo/python-list
Re: matrix multiplication
On Sun, 25 Feb 2018 18:52:14 -0500, Terry Reedy wrote: [...] > numpy has a matrix multiply function and now the '@' matrix multiply > operator. Yes, but what I was wondering is whether there's a faster way of multiplying each row (1x3) of a matrix by another matrix (3x3), compared to looping through the matrix row by row as shown in the code. Perhaps I'm not understanding how to use the broadcasting features of `matmul`. -- Seb -- https://mail.python.org/mailman/listinfo/python-list
Re: matrix multiplication
On Tue, 27 Feb 2018 12:25:30 +1300, Gregory Ewing wrote: > Seb wrote: >> I was wondering is whether there's a faster way of multiplying each >> row (1x3) of a matrix by another matrix (3x3), compared to looping >> through the matrix row by row as shown in the code. > Just multiply the two matrices together. > If A is an nx3 matrix and B is a 3x3 matrix, then C = A @ B is an nx3 > matrix where C[i] = A[i] @ B. > (This is a property of matrix multiplication in general, nothing > special about numpy.) I think that's only true if B is the same for every row in A. In the code I posted, B varies by row of A. -- Seb -- https://mail.python.org/mailman/listinfo/python-list
Re: matrix multiplication
On Tue, 27 Feb 2018 07:36:31 -0700, Ian Kelly wrote: > On Tue, Feb 27, 2018 at 4:08 AM, Peter Otten <__pete...@web.de> wrote: >> Seb wrote: >>> On Tue, 27 Feb 2018 12:25:30 +1300, >>> Gregory Ewing wrote: >>>> Seb wrote: >>>>> I was wondering is whether there's a faster way of multiplying >>>>> each row (1x3) of a matrix by another matrix (3x3), compared to >>>>> looping through the matrix row by row as shown in the code. >>>> Just multiply the two matrices together. >>>> If A is an nx3 matrix and B is a 3x3 matrix, then C = A @ B is an >>>> nx3 matrix where C[i] = A[i] @ B. >>>> (This is a property of matrix multiplication in general, nothing >>>> special about numpy.) >>> I think that's only true if B is the same for every row in A. In >>> the code I posted, B varies by row of A. >> Yeah, you would have to substitute the N 3x3 matrices with an Nx3x3 >> tensor, though I don't know if numpy provides an op such that >> Nx3 op Nx3x3 --> desired result >> or >> op(Nx3, Nx3x3) --> desired result > Nx1x3 @ Nx3x3 ought to do it, with the result being Nx1x3. That's right. I just tried this manipulation by replacing the last block of code in my example, from the line above `for` loop with: ---<cut here---start--->--- # Alternative using `np.matmul` uvw_alt = uvw.reshape((uvw.shape[0], 1, uvw.shape[1])) bmats = np.asarray(map(randint_mat, maxint)) uvw_rots_alt = np.matmul(uvw_alt, bmats).squeeze() ---<cut here---end->--- Interestingly, the time savings from IPython are not spectacular: %run -t -N100 loop_approach.py IPython CPU timings (estimated): Total runs performed: 100 Times : Total Per run User : 0.28 s, 0.00 s. System : 0.00 s, 0.00 s. Wall time: 0.28 s. %run -t -N100 matmul_approach.py IPython CPU timings (estimated): Total runs performed: 100 Times : Total Per run User : 0.17 s, 0.00 s. System : 0.00 s, 0.00 s. Wall time: 0.18 s. -- Seb -- https://mail.python.org/mailman/listinfo/python-list
generating list of files matching condition
Hello, Given a list of files: In [81]: ec_files[0:10] Out[81]: [u'EC_20160604002000.csv', u'EC_2016060401.csv', u'EC_20160604012000.csv', u'EC_20160604014000.csv', u'EC_2016060402.csv'] where the numbers are are a timestamp with format %Y%m%d%H%M%S, I'd like to generate a list of matching files for each 2-hr period in a 2-h frequency time series. Ultimately I'm using Pandas to read and handle the data in each group of files. For the task of generating the files for each 2-hr period, I've done the following: beg_tstamp = pd.to_datetime(ec_files[0][-18:-4], format="%Y%m%d%H%M%S") end_tstamp = pd.to_datetime(ec_files[-1][-18:-4], format="%Y%m%d%H%M%S") tstamp_win = pd.date_range(beg_tstamp, end_tstamp, freq="2H") So tstamp_win is the 2-hr frequency time series spanning the timestamps in the files in ec_files. I've generated the list of matching files for each tstamp_win using a comprehension: win_files = [] for i, w in enumerate(tstamp_win): nextw = w + pd.Timedelta(2, "h") ifiles = [x for x in ec_files if pd.to_datetime(x[-18:-4], format="%Y%m%d%H%M%S") >= w and pd.to_datetime(x[-18:-4], format="%Y%m%d%H%M%S") < nextw] win_files.append(ifiles) However, this is proving very slow, and was wondering whether there's a better/faster way to do this. Any tips would be appreciated. -- Seb -- https://mail.python.org/mailman/listinfo/python-list
Re: generating list of files matching condition
On Thu, 24 Nov 2016 10:18:21 +0100, Peter Otten <__pete...@web.de> wrote: > Is win_files huge? Then it might help to avoid going over the entire > list for every interval. Instead you can sort the list and then add to > the current list while you are below nextw. > My pandas doesn't seem to have Timedelta (probably it's too old), so > here's a generic solution using only the stdlib: [...] > PS: If the files' prefixes differ you cannot sort by name. Instead use > ec_files.sort(key=filename_to_time) > PPS: There is probably a way to do this by converting the list to a > pandas dataframe; it might be worthwhile to ask in a specialised > forum. Thank you, I learned a few things there (particularly nonlocal, since I'm still in Python 2.7)! -- Seb -- https://mail.python.org/mailman/listinfo/python-list
building numpy arrays with regular structure
Hello, Is there an easier way to write a numpy array with a regular structure? For example, an array with [0, 1] along the diagnal of one of the array dimensions, and zero elsewhere: zz = np.array([[[0, 1], [0, 0], [0, 0]], [[0, 0], [0, 1], [0, 0]], [[0, 0], [0, 0], [0, 1]]]) This one is not so big, but if it were, there must be a way to code this properly. Thanks, -- Seb -- https://mail.python.org/mailman/listinfo/python-list
parsing encrypted netrc file
Hello,
What's the pythonic way to do this without polluting the user's
directory with the decrypted file? I wrongly thought this should do it:
import os.path as osp
import gnupg
import netrc
import tempfile
gpg = gnupg.GPG()
with open(osp.expanduser("~/.authinfo.gpg"), "rb") as f:
with tempfile.NamedTemporaryFile("w+") as tf:
status = gpg.decrypt_file(f, output=tf.name)
info = netrc.netrc(tf.name)
which fails as the temporary file doesn't even get created.
--
Seb
--
https://mail.python.org/mailman/listinfo/python-list
Re: parsing encrypted netrc file
On Tue, 23 Jun 2020 00:40:28 +0100,
MRAB wrote:
> On 2020-06-22 23:38, Seb wrote:
>> Hello,
>> What's the pythonic way to do this without polluting the user's
>> directory with the decrypted file? I wrongly thought this should do
>> it:
>> import os.path as osp import gnupg import netrc import tempfile
>> gpg = gnupg.GPG()
>> with open(osp.expanduser("~/.authinfo.gpg"), "rb") as f: with
>> tempfile.NamedTemporaryFile("w+") as tf: status = gpg.decrypt_file(f,
>> output=tf.name) info = netrc.netrc(tf.name)
>> which fails as the temporary file doesn't even get created.
> Are you sure it doesn't get created?
Without using tempfile:
with open(osp.expanduser("~/.authinfo.gpg"), "rb") as f:
status = gpg.decrypt_file(f, output=".authinfo.txt")
info = netrc.netrc(".authinfo.txt")
I get the error:
NetrcParseError: bad follower token 'port' (.authinfo.txt, line 1)
which is interesting. The structure of ~/.authinfo.gpg is:
machine my.server.com login u...@foo.com password mypasswd port 587
so it seems this is not what netrc.netrc expects.
--
Seb
--
https://mail.python.org/mailman/listinfo/python-list
Re: parsing encrypted netrc file
On Tue, 23 Jun 2020 02:08:19 +0100, MRAB wrote: [...] > Here's a page I found about ".netrc": > https://ec.haxx.se/usingcurl/usingcurl-netrc > and here's a page I found about ".authinfo.gpg": > https://www.emacswiki.org/emacs/GnusAuthinfo > Can you see the subtle difference? Awww nuts, I do! What a pain, wouldn't it be nice if there was a standard on such things. Thanks for the pointer! -- Seb -- https://mail.python.org/mailman/listinfo/python-list
learning to use iterators
Hi, I'm fairly new to Python, and while trying to implement a custom sliding window operation for a pandas Series, I came across a great piece of code¹: >>> def n_grams(a, n): ... z = (islice(a, i, None) for i in range(n)) ... return zip(*z) ... I'm impressed at how succinctly this islice helps to build a list of tuples with indices for all the required windows. However, I'm not quite following what goes on in the first line of the function. Particulary, what do the parentheses do there? Thanks, +--- Footnotes ---+ ¹ http://sahandsaba.com/thirty-python-language-features-and-tricks-you-may-not-know.html#sliding-windows-n-grams-using-zip-and-iterators -- Seb -- https://mail.python.org/mailman/listinfo/python-list
Re: learning to use iterators
On Tue, 23 Dec 2014 12:23:45 -0700, Ian Kelly wrote: > The parentheses enclose a generator expression, which is similar to a > list comprehension [1] but produce a generator, which is a type of > iterator, rather than a list. > In much the same way that a list comprehension can be expanded out to > a for loop building a list, another way to specify a generator is by > defining a function using the yield keyword. For example, this > generator function is equivalent to the generator expression above: [...] Thank you, this exactly what I was missing. Happy holidays, -- Seb -- https://mail.python.org/mailman/listinfo/python-list
Re: learning to use iterators
Hi again, Thanks for your input; I'm starting to use generators to some extent. Say we have a series of numbers: x = randn(100) and values beyond some criteria should be considered as outliers, but only where there's at most 3 (or some other integer) consecutive values beyond the criteria. The outliers should then be replaced with linearly interpolated values using the non-outliers. outs = abs(x) > 0.5 # identify candidate outliers # where(outs) would tell the location of the outliers Here we could use something like: from itertools import groupby for val, g in groupby(outs): if outs: len(list(g))# just getting sequence length here to work with each outlier sequence, but then the indices in x are forgotten. They'd be needed to get the interpolates. I feel there's idioms that I should learn about to handle this situation more effectively. Cheers, -- Seb -- https://mail.python.org/mailman/listinfo/python-list
scalar vs array and program control
Hello, I'm fairly new to Python, struggling to write in a more object-oriented, functional style. I just wrote a function that takes two arrays representing sine (y) and cosine (x) angle coordinates, and returns the angle in degrees. I had initially written the function to take array-like arguments x/y, but I'd like to generalize and take scalars as well. However, the function has a subsetting operations, which don't work with scalars: vmag = np.sqrt((x ** 2) + (y ** 2)) ang = np.arctan2(y, x) ang[ang < 0] = ang[ang < 0] + (2 * np.pi) # output range 0 - 2*pi ang[vmag == 0] = 0 # when magnitude is 0 the angle is also 0 ang[ang == 0] = 2 * np.pi # convention If I want to take scalars x/y, I naively thought about implementing an if/else statement right before the subsetting operations. However, my intuition tells me there must be a proper object-oriented solution to this. Any tips appreciated. Cheers, -- Seb -- https://mail.python.org/mailman/listinfo/python-list
Re: Gmail eats Python
And for those interested in how I received Laura's message (the one I replied to): ---<cut here---start--->--- Path: news.gmane.org!not-for-mail From: Laura Creighton Newsgroups: gmane.comp.python.general Subject: Re: scalar vs array and program control Date: Sat, 25 Jul 2015 14:44:43 +0200 Lines: 44 Approved: n...@gmane.org Message-ID: <201507251244.t6pcihmd023...@fido.openend.se> References: <87vbd850qa@gmail.com><201507251101.t6pb1lqe021...@fido.openend.se> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1437828370 13839 80.91.229.3 (25 Jul 2015 12:46:10 GMT) X-Complaints-To: use...@ger.gmane.org NNTP-Posting-Date: Sat, 25 Jul 2015 12:46:10 + (UTC) Cc: Seb To: python-list@python.org Original-X-From: python-list-bounces+python-python-list=m.gmane@python.org Sat Jul 25 14:46:03 2015 Return-path: Envelope-to: python-python-l...@m.gmane.org Original-Received: from mail.python.org ([82.94.164.166]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1ZIyq2-0002GT-Ta for python-python-l...@m.gmane.org; Sat, 25 Jul 2015 14:46:03 +0200 Original-Received: from albatross.python.org (localhost [127.0.0.1]) by mail.python.org (Postfix) with ESMTP id 3mdnFQ2j3LzPgt for ; Sat, 25 Jul 2015 14:46:02 +0200 (CEST) X-Original-To: python-list@python.org Delivered-To: python-l...@mail.python.org Original-Received: from albatross.python.org (localhost [127.0.0.1]) by mail.python.org (Postfix) with ESMTP id 3mdnD81R8jzPZj for ; Sat, 25 Jul 2015 14:44:56 +0200 (CEST) X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'revision': 0.05; 'chunk': 0.07; 'exception.': 0.07; 'smallest': 0.07; 'thats': 0.07; 'valueerror:': 0.07; 'received:openend.se': 0.09; 'received:theraft.openend.se': 0.09; 'skip:t 60': 0.09; 'ignore': 0.14; 'things.': 0.15; '>try:': 0.16; 'from:addr:lac': 0.16; 'from:addr:openend.se': 0.16; 'from:name:laura creighton': 0.16; 'happily': 0.16; 'message-id:@fido.openend.se': 0.16; 'received:89.233': 0.16; 'received:89.233.217': 0.16; 'received:89.233.217.133': 0.16; 'received:fido': 0.16; 'received:fido.openend.se': 0.16; 'subject:array': 0.16; 'subject:program': 0.16; 'try/except': 0.16; 'laura': 0.18; 'stick': 0.18; 'try:': 0.18; 'cc:2**0': 0.20; 'posted': 0.21; 'work,': 0.21; 'pass': 0.22; 'errors': 0.23; 'forgot': 0.23; 'this:': 0.23; 'cc:addr:gmail.com': 0.24; 'written': 0.24; 'header :In-Reply-To:1': 0.24; 'skip:m 30': 0.27; 'e Original-Received: from localhost (HELO mail.python.org) (127.0.0.1) by albatross.python.org with SMTP; 25 Jul 2015 14:44:56 +0200 Original-Received: from theraft.openend.se (theraft.ipv6.openend.se [IPv6:2001:16d8:ffca::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.python.org (Postfix) with ESMTPS for ; Sat, 25 Jul 2015 14:44:55 +0200 (CEST) Original-Received: from fido.openend.se (r...@fido.openend.se [89.233.217.133]) by theraft.openend.se (8.14.4/8.14.4/Debian-4) with ESMTP id t6PCihR6024800 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Sat, 25 Jul 2015 14:44:45 +0200 Original-Received: from fido (lac@localhost [127.0.0.1]) by fido.openend.se (8.14.9/8.14.9/Debian-1) with ESMTP id t6PCihmd023479; Sat, 25 Jul 2015 14:44:43 +0200 In-Reply-To: Message from Laura Creighton of "Sat, 25 Jul 2015 13:01:21 +0200." <201507251101.t6pb1lqe021...@fido.openend.se> Content-ID: <23477.1437828283.1@fido> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.3.9 (theraft.openend.se [89.233.217.130]); Sat, 25 Jul 2015 14:44:45 +0200 (CEST) X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.20+ Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-requ...@python.org?subject=unsubscribe> List-Archive: <http://mail.python.org/pipermail/python-list/> List-Post: <mailto:python-list@python.org> List-Help: <mailto:python-list-requ...@python.org?subject=help> List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-requ...@python.org?subject=subscribe> Errors-To: python-list-bounces+python-python-list=m.gmane@python.org Original-Sender
debugging during package development
Hello, It seems too cumbersome to have to update `sys.path` to include the development tree of a package (and sub-packages) that's still very young. With lots of debugging to do, the last thing I'd want is to worry about the search path. So I've been searching for better ways to work, but I can't seem hit the right keywords and come with all sorts of tangentially related stuff. I'm sure there must be some tool that sets up the development environment when the package source is not on `sys.path`. Any advice on this topic would be appreciated. Cheers, -- Seb -- https://mail.python.org/mailman/listinfo/python-list
subdividing a rectangle using numpy
Hello, The code below is what I came up with to solve the problem: 1. We're given geographic coordinates for two opposite vertices of a rectangle. 2. If any of the sides of the rectangle is larger than some number of degrees, then subdivide the rectangle into squares/rectangles such that all sub-units have sides smaller than the specified amount. 3. Define each sub-unit by providing a list of all vertices, where the first and last vertices are identical so as to close the polygon. I've ignored the "if" part of the problem to simplify. The key to my solution was to use numpy's meshgrid to generate the coordinates for defining the sub-units. However, it seems awfully complex and contrived, and am wondering if there's a simpler solution, or perhaps some package offers this functionality. I couldn't find any, so any tips appreciated. ---<cut here---start--->--- # Let's say we have these coordinates lons = [-96, -51.4] lats = [60, 72] # And we want to create 10x10 (max) degree polygons covering the rectangle # defined by these coordinates step = 10 # Calculate how many samples we need for linspace. The ceiling is required # to cover the last step, and then add two to accomodate for the inclusion # of the end points... what a pain. xn = np.ceil((lons[1] - lons[0]) / step) + 2 yn = np.ceil((lats[1] - lats[0]) / step) + 2 xgrd = np.linspace(lons[0], lons[1], xn) ygrd = np.linspace(lats[0], lats[1], yn) # Create grids of longitudes and latitudes with dimension (yn, xn). The # elements of the longitude grid are the longitude coordinates along the # rows, where rows are identical. The elements of the latitude grid are # the latitude coordinates along the columns, where columns are identical. longrd, latgrd = np.meshgrid(xgrd, ygrd, sparse=False) for i in range(int(xn) - 1): for j in range(int(yn) - 1): print [(longrd[j, i], latgrd[j, i]), # lower left (longrd[j, i + 1], latgrd[j, i]), # lower right (longrd[j, i + 1], latgrd[j + 1, i]), # upper right (longrd[j, i], latgrd[j + 1, i]), # upper left (longrd[j, i], latgrd[j, i])] # close at lower left ---<cut here-------end->--- -- Seb -- https://mail.python.org/mailman/listinfo/python-list
IPython in Emacs
Hi,
Please excuse the slightly off-topic query. I'm learning Python, using
the IPython (0.13) shell, and wanted to run it from Emacs 24. AFAICT,
python.el is the most actively developed library, and is included in
Emacs. How do experienced Python programmers set up their python.el to
make the best of it? I've done it following the recommendations given
in the library¹:
(setq
python-shell-interpreter "ipython"
python-shell-interpreter-args ""
python-shell-prompt-regexp "In \\[[0-9]+\\]: "
python-shell-prompt-output-regexp "Out\\[[0-9]+\\]: "
python-shell-completion-setup-code
"from IPython.core.completerlib import module_completion"
python-shell-completion-module-string-code
"';'.join(module_completion('''%s'''))\n"
python-shell-completion-string-code
"';'.join(get_ipython().Completer.all_completions('''%s'''))\n")
but this may be a little outdated as it refers to IPython 0.11.
Thanks,
Seb
+--- Footnotes ---+
¹ Ignored recommended setting for `python-shell-interpreter-args'
--
http://mail.python.org/mailman/listinfo/python-list
Re: IPython in Emacs
On Wed, 24 Apr 2013 21:38:04 -0700 (PDT), rusi wrote: > There were some ipython+emacs+windows bugs: > https://bugs.launchpad.net/ipython/+bug/290228 > Last I tried nearly 2 years, they were still there > http://groups.google.com/group/comp.lang.python/browse_thread/thread/36e757567f28368e On Debian here, so not a problem. Thanks, -- Seb -- http://mail.python.org/mailman/listinfo/python-list
Re: IPython in Emacs
On Thu, 25 Apr 2013 06:54:33 -0700 (PDT), rusi wrote: > On Apr 25, 6:01 pm, Seb wrote: >> On Wed, 24 Apr 2013 21:38:04 -0700 (PDT), >> rusi wrote: >> > There were some ipython+emacs+windows bugs: >> >https://bugs.launchpad.net/ipython/+bug/290228 > Last I tried nearly >> 2 years, they were still there >> >http://groups.google.com/group/comp.lang.python/browse_thread/thread/... >> On Debian here, so not a problem. > Ok me too :-) > A quick try of your startup code did not work for me. There is a > perennial clash in emacs between python.el and python- mode.el -- I > guess its that. Which do you use? The first. I don't have anything python-related in ~/.emacs other than what I showed. Works fine for me, except for weird things like multiple input prompts in the IPython shell at startup, which seem to come from each statement in the variables I showed. Similar things happen when sending code from the script buffer. > What are your commands for eval-buffer and start-interpreter? I don't understand what you're asking. I evaluate the script buffer with `python-shell-send-buffer' and start IPython with `run-python'. -- Seb -- http://mail.python.org/mailman/listinfo/python-list
getting quick arp request
Hello, What I need : I need to write a scanner that test all the IP adresses that repond on a given port. The Ip list is of roughly of length 200. I need to get the response every 60 seconds (or better). I would prefer first not to use nmap. Configuration : * Python 2.4.1. To test what is going on I use ethereal. I am using winXP pro on a 2GHZ P4 and 512 Mo. *** Problem : *** I tried to implement a simplistic threaded version where each thread is opening a blocking socket on the IP and port. I have monitored using etherereal that I get one arp query every second roughly. I am expecting a speed on the same oder of magnitude as the one that one can get from a standard IP/port scanner. To compare, I have used angry Ip scanner and I have seen that roughly 200 arp request where sent in 20 seconds. *** Also : *** I have also considered using some asynchrone connection but AFAIK you need first to open the socket and so to use the arp protocol. Thanks I advance for your help. Sebastien. * Code sample : * # Sebastien 6/9/2006 for testing purposes import time import Queue from threading import * import threading import socket try : import psyco psyco.full() except : pass class socket_test (Thread): def __init__ (self,adresse): Thread.__init__(self) self.PORT=21 self.adresse=str(adresse) print "in thread adresse = ", self.adresse self.service=[] self.start() def run(self) : service_unit=socket.socket(socket.AF_INET, socket.SOCK_STREAM) service_unit.setblocking(1) print "socket Ip = ",self.adresse try : service_unit.connect((str(self.adresse), self.PORT)) except Exception,e: print "exception ",e self.service.append(service_unit) class groupe_thread : def __init__(self,liste): self.liste=liste def go(self): print "self.liste = ",self.liste for el in self.liste : print "go starting thread on : ",el s=socket_test(el) liste=[] base ="192.168.3." rang=range(1,50) for r in rang: add=base+str(r) liste.append(add) a=groupe_thread(liste) ut= a.go() print "the end (main) .." -- http://mail.python.org/mailman/listinfo/python-list
Re: getting quick arp request
Hi Ben, I am indeed using XP SP2. I have checked on the event viewer and I have not seen the event 4226. Besides I also run on the same PC angry Ip scanner 2.21. Checking using ethereal the arp request are all dispatched quit quickly (see my mail above). Thanks for the advice anyway. Sebastien. Ben Sizer wrote: > seb wrote: > > I need to write a scanner that test all the IP adresses that repond on > > a given port. > ... > > I am using winXP pro on a 2GHZ P4 and 512 Mo. > > If you have XP Service Pack 2, it cripples port-scanning as part of a > 'security' fix. Broadly speaking, it limits the rate at which you can > make connections at the OS level; this will show up as event 4226 in > the Event Viewer if it affects you. > > -- > Ben Sizer -- http://mail.python.org/mailman/listinfo/python-list
Re: getting quick arp request
Hi Ben, I am indeed using XP SP2. - Some more info : - 1) I have checked on the event viewer and I have not seen the event 4226 while I have run the code sample above. 2) I can still see this error (4226) recently In the log so that I must have bumped against this limit trying to put pull this out. 3) I have installed today process explorer (from sysinternals). I am not completly used to it but you can have a look at the TCP/IP connections opened by the processes. It appears that I have alwyas 10 connections opened (and the IP adresses progress durning the scan from Ip adresse 192.168.3.1 -> 254). 4) Besides I also run on the same PC angry Ip scanner 2.21. Checking using ethereal the arp request are all dispatched quit quickly (see my mail above). NEW RESULT : --- Something is limiting the TCP/IP connections from my python program at 10 maximum at the same time. I do not see this limit in my code. I did not bumped over the 4226 error. => Where does this limit come from. => How can I overcome it. Thanks for the advice anyway. Sebastien. Ben Sizer wrote: > seb wrote: > > I need to write a scanner that test all the IP adresses that repond on > > a given port. > ... > > I am using winXP pro on a 2GHZ P4 and 512 Mo. > > If you have XP Service Pack 2, it cripples port-scanning as part of a > 'security' fix. Broadly speaking, it limits the rate at which you can > make connections at the OS level; this will show up as event 4226 in > the Event Viewer if it affects you. > > -- > Ben Sizer -- http://mail.python.org/mailman/listinfo/python-list
Re: getting quick arp request
Thank you all for the reply, ** More tests : *** 1) I tried to input the D-word with the parameters and I did not see anychanged (checked with process explorer. The limit of the simultaneous connexion is always 10. 2) I have applied the patch from http://www.lvllord.de/?lang=en&url=downloads . I could see that this improved the simultaneous sockets up to roughly 50. This is enough for me. 3) Since during the scan the first protocol used (and packet capteures) is using the arp protocol, the subject may be indeed a misnomer. Question : * 1) I am not fully confident to apply the patch from http://www.lvllord.de/?lang=en&url=downloads .on computers other than mine. Is there another solution ? 2) Still without the above patch on windows, the software "angry ip scan" for example managed to output a lot of more socket connection. How is it possible ? Regards. Sebastien. Steve Holden wrote: > Richard Brodie wrote: > > "Steve Holden" <[EMAIL PROTECTED]> wrote in message > > news:[EMAIL PROTECTED] > > > > > >>Is it relevant to point out that the ARP protocol is a connectionless > >>network-layer > >>protocol. > > > > > > Not really, since the program uses normal TCP socket connections. > > The feature is working exactly as designed - to slow down TCP scans. > > The arp requests are just a consequence of the TCP scan. > > > > > Ah. Right. Now you mention that (and force me to read the code :-) I see > it's a horizontal scan of the FTP service port, and the subject line is > really a misnomer. Thanks. > > regards > Steve > -- > Steve Holden +44 150 684 7255 +1 800 494 3119 > Holden Web LLC/Ltd http://www.holdenweb.com > Skype: holdenweb http://holdenweb.blogspot.com > Recent Ramblings http://del.icio.us/steve.holden -- http://mail.python.org/mailman/listinfo/python-list
what is the best practice to separate Pygtk and long running thread code
Hi,
I am using pygtk for the first times.
I am wondering what would be the best "pattern" to interface pygtk with
a thread.
The thread is collecting informations (over the network for example) or
is doing some long calculations.
I would like also to separate the gui part to the action part so that I
should be easier to maintain.
What should be the best practice in order to achieve this ?
***
What I use now is :
1)
the gui is launching the "action" evrey 1 sec and is checking for
message using a queue every second also.
gui.py
etc ...
class window1(SimpleGladeApp):
[EMAIL PROTECTED] class window1 }
[EMAIL PROTECTED] init window1.__init__ {
def __init__(self, path='gui.glade',
root='window1',
domain=app_name, kwargs={}):
path = os.path.join(glade_dir, path)
SimpleGladeApp.__init__(self, path, root, domain, **kwargs)
self.q=Queue.Queue()
self.action=act.action(self.q)
gobject.timeout_add (1000,self.action.go) # this is the action
asked
gobject.timeout_add (1000,self.process) # check if a new message is
available in the queue
def process (self):
dir (self.q)
if self.q.empty() == False :
print "from main ",self.q.get()
return True
...etc
2)
The action part is making somehow the interface between the running
thread and the gui :
It puts the new information in the queue for the gui to process it
later.
action.py
import os
import gui
import time
import Queue
import Spethread # thread that always run in the background
class Singleton(object):
_instance = None
def __new__(cls, *args, **kwargs):
if not cls._instance:
cls._instance = super(Singleton, cls).__new__(cls, *args,
**kwargs)
return cls._instance
class action(Singleton):
def __init__(self, queue):
self.q=queue
self.thread_seb=Spethread.worker()
self.thread_seb.start()
self.go()
def go(self):
if self.thread_seb:
reponse=self.thread_seb.status()
self.q.put(reponse)
else :
self.q.put("le thread n'existe plus ")
return True
def stop(self):
self.thread_seb.die()
3)
The thread part is "just" a thread that should from time to time
time.Sleep in order for action to query if it has some new messages.
Spethread.py
import threading
import time
class worker(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.value=0
self.go_on=1
def run(self):
print "self.go_on=",self.go_on
while self.go_on == 1:
self.value=int(time.time())
res=2
for i in range(0, 100):
res=res^i
print res
time.Sleep(0.01)
def status(self):
return self.value
def die(self):
print "die request"
self.go_on=0
Thanks in advance for sharing this informations.
Sebastien.
PS :
the entire gui.py
[EMAIL PROTECTED] python gui.py {
[EMAIL PROTECTED] header gui.py {
#!/usr/bin/env python
# -*- coding: UTF8 -*-
# Python module gui.py
# Autogenerated from gui.glade
# Generated on Wed Sep 20 22:03:00 2006
# Warning: Do not modify any context comment beginning with # @--
# They are required to keep user's code
# Doing so will make it unable for kefir to help you further manage
your project
[EMAIL PROTECTED] header gui.py }
[EMAIL PROTECTED] app gui {
import os
import gobject
import gtk
import action as act
import Queue
from SimpleGladeApp import SimpleGladeApp, bindtextdomain
app_name = 'gui'
app_version = '0.0.1'
glade_dir = ''
locale_dir = ''
bindtextdomain(app_name, locale_dir)
[EMAIL PROTECTED] app gui }
[EMAIL PROTECTED] window window1 {
[EMAIL PROTECTED] class window1 {
class window1(SimpleGladeApp):
[EMAIL PROTECTED] class window1 }
[EMAIL PROTECTED] init window1.__init__ {
def __init__(self, path='gui.glade',
root='window1',
domain=app_name, kwargs={}):
path = os.path.join(glade_dir, path)
SimpleGladeApp.__init__(self, path, root, domain, **kwargs)
self.q=Queue.Queue()
self.action=act.action(self.q)
gobject.timeout_add (1000,self.action.go)
gobject.timeout_add (1000,self.process)
[EMAIL PROTECTED] init window1.__init__ }
[EMAIL PROTECTED] new window1.new {
def new(self):
print 'A new %s has been created' % self.__class__.__name__
[EMAIL PROTECTED] new window1.new }
[EMAIL PROTECTED] custom window1 {
# Write your own methods here
[EMAIL PROTECTED] custom window1 }
[EMAIL PROTECTED] callback window1.on_bu
Re: what is the best practice to separate Pygtk and long running thread code
Hi Thomas,
I am running WinXP so that casting processes and getting their results
is not so convenient.
I have tested idle add and it does the job : the thread is running
whenever there is no activity on the gui.
I still do not understand how it can be so responsive 'cause the thread
I am using at the moment do not have any time.sleep(ing).
I am afraid I have still to use an intermediate class between the
computing thread and the gui to send data between the two of them but
now about all the computing time is devoided to the thread.
Thanks a lot !!!
Seb.
ps : the only mod that I have done to the gui.py in the init is
def __init__(self, path='gui.glade',
root='window1',
domain=app_name, kwargs={}):
path = os.path.join(glade_dir, path)
SimpleGladeApp.__init__(self, path, root, domain, **kwargs)
self.q=Queue.Queue()
self.action=act.action(self.q)
gobject.idle_add(self.action.go)
#gobject.timeout_add (1000,self.action.go)
gobject.timeout_add (1000,self.process)
Thomas Guettler wrote:
> seb wrote:
>
> > Hi,
> >
> > I am using pygtk for the first times.
> >
> > I am wondering what would be the best "pattern" to interface pygtk with
> > a thread.
> >
> > The thread is collecting informations (over the network for example) or
> > is doing some long calculations.
>
> Hi,
>
> I would use several *processes*. If your scripts runs on Unix/Linux
> you can use select() on the filedescriptors of the processes you created
> with popen. On Windows you need to poll them, but maybe this is
> better than threads, too. With idle_add you can get data from
> the subproceses. It gets called if there are no actions in the
> event-loop.
>
> HTH,
> Thomas
>
> --
> Thomas Güttler, http://www.thomas-guettler.de/ http://www.tbz-pariv.de/
> E-Mail: guettli (*) thomas-guettler + de
> Spam Catcher: [EMAIL PROTECTED]
--
http://mail.python.org/mailman/listinfo/python-list
Re: what is the best practice to separate Pygtk and long running thread code
Hi, I am running on WinXP so that is not so convenient to communicate between separate processes. In the past I have used blocking files but it may be a slow communication way for some applications. It may also be too much depending on the disk on which the program is run (network drives with different rights). It seems that RPC call would do the job but I would have liked something simplier (although I did not tried it). The best thing would be to have a queue feature that would be be shared between processes but as far as I know It does not exists in python. Thanks. Seb Do you know of some features like a queue that [EMAIL PROTECTED] wrote: > seb wrote: > > Hi, > > > > I am using pygtk for the first times. > > > > I am wondering what would be the best "pattern" to interface pygtk with > > a thread. > > > > The thread is collecting informations (over the network for example) or > > is doing some long calculations. > > It sounds like you don't need to share all your memory--any time you're > trying to "seperate" threads, you probably want to be using processes > instead. -- http://mail.python.org/mailman/listinfo/python-list
convert frames from mpeg to array
Hi, I need to convert each frame from a movie (mpeg for example) to an array (in order to do some computation) and then back to a video format mpeg for example. Do you know of any tools ? The equipment : I got myself an IP axis camera with wich I am playing to trying to do some shape recognition. My PC is running winXP SP2. The plan is : -- 1) to manage to record the movies to my hard disk (in mpeg format for example) 2) to analyse the films in order to tests various algorythms / computation 3) to generate the transformed frame back to mpeg (or any format). What is already done is : - 1) record to hard disk is easily donne using vlc for example vlc "http://192.168.0.90/axis-cgi/mjpg/video.cgi?fps=30&nbrofframes=0"; --sout file/ts:test.mpg The problem is point 2 - I have tried to use pygame (on winXP) but it seems that it does not suppport movies. Pycar does not run on my PC. I have looked for some other tools but without any success. There is possibily a python binding to vlc but I am not sure of what can be done with it. I thank you in advance for taking the time to read this mail. Best regards. Seb. -- http://mail.python.org/mailman/listinfo/python-list
line duplication using logging to file
Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.
My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)
I am using python 2.4 on WinXP SP2.
Do you have any idea ? Thanks in advance.
Seb.
*
The very simple "logging program ":
*
import logging, logging.handlers
import time
def write_log(level, message):
# Utilisation de l'API pour le Handler global
print "time.asctime()",time.asctime(),"received level=",level,"message
=",message
nom_logger="main_log_file"
logger=logging.getLogger(nom_logger)
logger.setLevel(logging.DEBUG)
prefix = "pix_main_log_file"
#fh=logging.handlers.RotatingFileHandler(prefix + "_log.txt", 'a',
100,10)
fh=logging.FileHandler("main_log.txt")
fh.setLevel(logging.DEBUG)
#formater = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s
- %(message)s")
formater = logging.Formatter("%(asctime)s - %(levelname)s -
%(message)s")
fh.setFormatter(formater)
logger.addHandler(fh)
#logger.info("* debut")
#message = main(url,conf_file,logger)
#message="hello seb"
if str(level).lower() == "info" :
print "logger info"
logger.info(str(message))
elif str(level).lower() =="error":
print "logger error"
logger.error(str(message))
elif str(level).lower()=="warning" :
print "logger warning"
logger.warning(str(message))
elif str(level).lower() =="critical":
print "logger critical"
logger.critical(str(message))
elif str(level).lower() == "exception":
print "logger exception"
logger.exception(str(message))
else :
logger.info("niveau inconnu "+str(message))
print "_",message
#print dir(logger)
return
*
Example of the log file with duplicated line.
**
2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___init_rs232initrs232_openCOM1
2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___runthread lance
2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___runthread lance
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents an
Re: line duplication using logging to file
Hi,
Thanks for the help.
Meanwhile I have written the logging function from scratch and it works
without the multiple lines.
This means that the multiple line copy is not due to the multiple
processes (or thread) trying to access the log file but to something
else.
Thanks.
Sebastien.
the new function :
import logging
import time
def write_log(level, message):
nom_logger="main_log_file.txt"
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(levelname)s %(message)s',
filename=nom_logger,
filemode='w')
if str(level).lower() == "info" :
logging.info(str(message))
elif str(level).lower() =="error":
logging.error(str(message))
elif str(level).lower()=="warning" :
logging.warning(str(message))
elif str(level).lower() =="critical":
logging.critical(str(message))
elif str(level).lower() == "exception":
logging.exception(str(message))
else :
logging.INFO(str(message))
return
Paul McGuire a écrit :
> "seb" <[EMAIL PROTECTED]> wrote in message
> news:[EMAIL PROTECTED]
> > Hi, I am writing to a file some basic information using the logging
> > module. It is working but in the log file some line are printed
> > several time. I had put some print debugging messages in the logging
> > function (so they appear on the consile) and they are called once only.
> > Obviously there is some understantding of the logging module that I am
> > missing.
> >
> > My simple logging program (see below) is called by several processes.
> > In this way I can collect the information from various sources (and not
> > use the network enabled logging module)
> >
> > I am using python 2.4 on WinXP SP2.
> >
> > Do you have any idea ? Thanks in advance.
> >
> > Seb.
> >
>
> A quick tally of log messages by timestamp and comment gives this data:
>
> ('2007-01-08 18:26:19,578', '___init_rs232initrs232_openCOM1') : 1
> ('2007-01-08 18:26:19,578', '___runthread lance') : 2
> ('2007-01-08 18:26:32,015', '___test1TEST 1 = OK') : 3
> ('2007-01-08 18:26:42,483', '___test1TEST 1 = OK') : 4
> ('2007-01-08 18:26:53,750', '___test1TEST 1 = OK') : 5
> ('2007-01-08 18:27:03,092', '___test1TEST 1 = OK') : 6
> ('2007-01-08 18:27:13,671', '___test1TEST 1 = OK') : 7
> ('2007-01-08 18:27:14,796', '___run___fin dans le run car continue = 0') : 8
> ('2007-01-08 18:27:14,890', "___stopthread demande d'arret") : 9
> ('2007-01-09 08:51:26,562', '___init_rs232initrs232_openCOM1') : 1
> ('2007-01-09 08:51:26,733', '___runthread lance') : 2
> ('2007-01-09 08:51:39,453', '___test1TEST 1 = OK') : 3
> ('2007-01-09 08:51:48,280', '___test1TEST 1 = OK') : 4
> ('2007-01-09 08:51:58,750', '___test1TEST 1 = OK') : 5
> ('2007-01-09 08:52:09,812', '___test1TEST 1 = OK') : 6
> ('2007-01-09 08:52:19,078', '___test1TEST 1 = OK') : 7
> ('2007-01-09 08:52:22,078', '___run___fin dans le run car continue = 0') : 8
> ('2007-01-09 08:52:22,125', "___stopthread demande d'arret") : 8
> ('2007-01-09 08:52:22,125', "___stopthread demande d'arret ") : 1
>
> Does this suggest anything to you?
>
> -- Paul
>
>
> (BTW, here is the pyparsing program I used to do this analysis)
>
> from pyparsing import
> Word,nums,Combine,alphas,oneOf,Literal,SkipTo,restOfLine
>
> # create pyparsing grammar definition for a log line
> date = Word(nums,exact=4)+'-'+Word(nums,exact=2)+'-'+Word(nums,exact=2)
> time = Word(nums,exact=2)+':'+Word(nums,exact=2)+':'+Word(nums,exact=2)+ \
> ','+Word(nums,exact=3)
> timestamp = Combine(date + ' ' + time)
> severity = oneOf( ["INFO","WARNING"] ) # not complete, but enough for this
> data
> backslash = Literal("\\")
> fileref = Combine(Word(alphas,exact=1)+":" + backslash + SkipTo(".py") +
> ".py")
> logline = ( timestamp.setResultsName("timestamp") + "-" +
> severity.setResultsName("severity") + "-" +
> fileref.setResultsName("fileref") +
> restOfLine.setResultsName("comment") )
>
> # create list of ParseResults, with addressable log line elements by results
> name
> logEntries = [ logline.parseString(line) for line in logdata ]
>
> # tally up log lines by timestamp and comment
> tallyByTimestamp = {}
> for entry in logEntries:
> tallyKey = (entry.timestamp, entry.comment)
> tallyByTimestamp[tallyKey] = tallyByTimestamp.get(tallyKey,0) + 1
>
> for ts in sorted( tallyByTimestamp.items() ):
> print "%s : %d" % ts
--
http://mail.python.org/mailman/listinfo/python-list
connection to server not accepted (but no error) only after several hours
Hi, this simple server (time protocol) does not respond after a few
hours, even when it is restarted. The behaviour looks to me like a
firewall blocking but I have desabled the firewall.
Using Netstat - a I find the server listed when it is running and not
listed when I stop it.
The server starts whith no error. But after about 6-8 hours there is no
way to make it respond again (even when it is restarted).
Once it is blocked, I killes it, wait for about 15 minutes without
running the server and then started again but this gives me the same
behaviour (no response).
The solution at this time is to reboot windows !!!
I have also tried to run it twice in order to get the exception that
the port is already used and an exeption is raised.
I have tried from another computer to connect to mine but it does not
manage.
When I change the port in the server it responds immediatly.
I am running winXP SP2, python 2.4.
Do you have any clues ?
Thanks in advance.
Seb.
PS : I am calling this server from another program and make it run in a
thread.
Below is the standalone version (wich is adpated from effbot site).
import socket
import struct, time
import threading
import os
import appel_log2 as appel_log
import sys
class TimeServer(threading.Thread) :
def __init__(self):
nom_function_actuelle= str(sys._getframe().f_code.co_filename)
+"___"+str(sys._getframe().f_code.co_name)
try :
threading.Thread.__init__(self)
self.log_file="timeserverlog.txt"
self.PORT=37
self.TIME1970=2208988800L
self._continue=1
self.time_shift=0
message=nom_function_actuelle+"\t "+"STARTED OK "
appel_log.write_log("info",message)
except Exception, e :
message=nom_function_actuelle+"\t "+"PB:::"+str(e)
print message
appel_log.write_log("warning",message)
def set_log_file(self, file):
nom_function_actuelle= str(sys._getframe().f_code.co_filename)
+"___"+str(sys._getframe().f_code.co_name)
if os.path.exists(file):
pass
else :
f=open(file,"w")
f.close()
self.log_file=file
print "log file ",self.log_file
self.log_file=file
def reset_log_file(self):
nom_function_actuelle= str(sys._getframe().f_code.co_filename)
+"___"+str(sys._getframe().f_code.co_name)
print "resetting log file "
if os.path.exists(self.log_file):
f=open(self.log_file,"w")
f.close()
def set_time_shift(self,time_shift):
self.time_shift=time_shift
def run(self):
nom_function_actuelle= str(sys._getframe().f_code.co_filename)
+"___"+str(sys._getframe().f_code.co_name)
socket.timeout(1)
service=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
service.bind(("", self.PORT))
service.listen(1)
print "listening on port", self.PORT
while self._continue==1 :
channel, info = service.accept()
print "connection from", info
message=str(time.time())+"\t"+str(time.asctime())+"\t"+str(info)+"\n"
g=open(self.log_file,"a")
g.write(message)
g.close()
t = int(time.time()) + self.TIME1970 + self.time_shift
t = struct.pack("!I", t)
channel.send(t) # send timestamp
channel.close() # disco
m=nom_function_actuelle+" response OK "+str(info)
appel_log.write_log("info",m)
#print "apres m "
print "time server self_continue=0"
appel_log.write_log("warning",nom_function_actuelle+"\t
self._continue ="+str(self._continue))
print "sortie du thread"
def main() :
a=TimeServer()
a.start()
a.set_log_file("log_nw.txt")
a.reset_log_file()
while 1==1 :
time.sleep(10)
#a._continue=0
pass
time.sleep(2)
main()
--
http://mail.python.org/mailman/listinfo/python-list
Re: connection to server not accepted (but no error) only after several hours
Hi Dennis,
I am using indeed using python logging in the appel_log2 module.
But I wanted to keep it extremly simple while accepting connection from
different processes or thread.
Regarding my main problem, I did some more testing :
1) I have enabled one time server that can be run as a service (R C C
time server) and this service is responding correctly, when at the same
time (before I kill it ) the python time server is not responding.
2) I have also tried two python time server downloaded from effbot
site. Both are not responding after the "non response from the time
server I rn" even for the first interrogation once they are started.
(Of course I have killed my time server when I run a new one).
Same behaviour no response given but no error neither.
3)
If I try to start the R c c time server when the python time server is
running there is an error (from the rcc service)
The python time server program is really bound to port 37.
if we look at the previous tests :
4)
The same python program runned on port 38 aftter the blocking is
working properly from the start.
5)
There is no exception on the python time server whether it is
responding or not.
--
partial conclusion
-
It is only python programs that are listening from the port 37 that are
blocked at a certain time.
How is it possible (without firewall enabled) ?
Thanks .
Sebastien.
Dennis Lee Bieber a écrit :
> On 16 Jan 2007 07:39:35 -0800, "seb" <[EMAIL PROTECTED]>
> declaimed the following in comp.lang.python:
>
> >
> > Once it is blocked, I killes it, wait for about 15 minutes without
> > running the server and then started again but this gives me the same
> > behaviour (no response).
> >
> Can't help with the lock-up problem... but...
> >
> > import socket
> > import struct, time
> > import threading
> > import os
> > import appel_log2 as appel_log
> > import sys
> >
> >
> > class TimeServer(threading.Thread) :
> > def __init__(self):
> > nom_function_actuelle= str(sys._getframe().f_code.co_filename)
> > +"___"+str(sys._getframe().f_code.co_name)
> > try :
> > threading.Thread.__init__(self)
> > self.log_file="timeserverlog.txt"
> > self.PORT=37
> > self.TIME1970=2208988800L
> > self._continue=1
> > self.time_shift=0
> > message=nom_function_actuelle+"\t "+"STARTED OK "
> > appel_log.write_log("info",message)
>
> Have you considered using Python's logging module?
> > while 1==1 :
>
> Redundant test...
>
> In all historical versions of Python
>
> while 1:
>
> sufficed, and newer versions have
>
> while True:
> --
> WulfraedDennis Lee Bieber KD6MOG
> [EMAIL PROTECTED] [EMAIL PROTECTED]
> HTTP://wlfraed.home.netcom.com/
> (Bestiaria Support Staff: [EMAIL PROTECTED])
> HTTP://www.bestiaria.com/
--
http://mail.python.org/mailman/listinfo/python-list
Re: connection to server not accepted (but no error) only after several hours
Hi Dennis,
I think I have some new informations.
My system is "blocked" now but the following change make it work again
!!!
I will test it for tonight to be sure of the improvements.
I changed :
service.bind(("", self.PORT))
to
service.bind((socket.gethostname(), self.PORT))
The strange thing is that using the "" it works for a few hours.
Regards.
Sebastien.
Dennis Lee Bieber a écrit :
> On 17 Jan 2007 00:08:52 -0800, "seb" <[EMAIL PROTECTED]>
> declaimed the following in comp.lang.python:
>
> >
> > 1) I have enabled one time server that can be run as a service (R C C
> > time server) and this service is responding correctly, when at the same
> > time (before I kill it ) the python time server is not responding.
> >
>
> What behavior do you see if you don't run them as background
> services, but rather from a regular console login?
>
> > 2) I have also tried two python time server downloaded from effbot
> > site. Both are not responding after the "non response from the time
> > server I rn" even for the first interrogation once they are started.
> > (Of course I have killed my time server when I run a new one).
> > Same behaviour no response given but no error neither.
> >
> If you've now got a total of three programs that are not reporting
> error conditions, I'd suspect there is something else wrong in the
> system...
>
> > It is only python programs that are listening from the port 37 that are
> > blocked at a certain time.
> > How is it possible (without firewall enabled) ?
> >
> What response do you get from the clients attempting to connect to
> this server? (I'd expect a either a flat out "denied" or, for a
> stealthed firewall, a timeout with no response).
>
>
> You also have a race condition in your log-file...
>
> > a.start()
> > a.set_log_file("log_nw.txt")
> > a.reset_log_file()
>
> It is possible that the thread gets a few connections between the
> .start() and the .set_log_file() and logs them to the default file name.
> Also, it is possible for connections to be logged between the
> .set_log_file() and the .reset_log_file() (where you wipe out the
> contents of the log file).
>
> I'd suggest putting the .start() call third in that list. That way
> you create the thread object, but it is not running. Change the log file
> name, wipe out any old contents, and THEN start the thread running.
>
> My only other comment would be to add a few wolf-fences... Print
> statements (if running in a console), or more logging messages (you
> might want to make a method out of that internal logging so all you code
> is, say
>
> self.slog("message")
>
> and "slog" does that time stamping, and file open/close...
>
> By logging each main step (accept, send, close) you might find where
> it stops.
> --
> WulfraedDennis Lee Bieber KD6MOG
> [EMAIL PROTECTED] [EMAIL PROTECTED]
> HTTP://wlfraed.home.netcom.com/
> (Bestiaria Support Staff: [EMAIL PROTECTED])
> HTTP://www.bestiaria.com/
--
http://mail.python.org/mailman/listinfo/python-list
Brussels Python Interest/Users Group
Hi Pythonistas, Is anyone interested in forming a Brussels(Belgium) area Python User Group ? I am not aware of any python focused group in this area. Language could be whatever fits the bill (English/French/Dutch/...) Focus would be python, ironpython/silverlight, scipy, ORMs, web frameworks, and whatever is of interest to us. Interested ? Please post on this thread. Seb -- http://mail.python.org/mailman/listinfo/python-list
xml menu pygtk
I'm trying to construct a menu from an xml file. However my recursive algorithm isn't doing what I want it too. I've been starring at this for too long. Any help appreciated :) I get the following error but the problem is more of a logical nature. ./gnomeAppletMenu.py:40: GtkWarning: gtk_menu_shell_insert: assertion `GTK_IS_MENU_ITEM (child)' failed menu_bar.append(menu) I'm pretty sure I fuck up around line 27-30 xml file: http://pastie.org/480045 python code: http://pastie.org/480042 best regards, Seb -- http://mail.python.org/mailman/listinfo/python-list
Re: xml menu pygtk
On May 16, 4:20 pm, MRAB wrote: > Seb wrote: > > I'm trying to construct a menu from an xml file. However my recursive > > algorithm isn't doing what I want it too. I've been starring at this > > for too long. Any help appreciated :) > > > I get the following error but the problem is more of a logical nature. > > ./gnomeAppletMenu.py:40: GtkWarning: gtk_menu_shell_insert: assertion > > `GTK_IS_MENU_ITEM (child)' failed > > menu_bar.append(menu) > > > I'm pretty sure I fuck up around line 27-30 > > > xml file: > >http://pastie.org/480045 > > > python code: > >http://pastie.org/480042 > > The traceback suggests to me that you can add only a 'MenuItem' to a > 'MenuBar', but create_menu() is returning a 'Menu'. Doesn't a MenuBar consists of Menu's? best regards, Seb -- http://mail.python.org/mailman/listinfo/python-list
Re: xml menu pygtk
On May 16, 5:04 pm, MRAB wrote:
> Seb wrote:
> > On May 16, 4:20 pm, MRAB wrote:
> >> Seb wrote:
> >>> I'm trying to construct a menu from an xml file. However my recursive
> >>> algorithm isn't doing what I want it too. I've been starring at this
> >>> for too long. Any help appreciated :)
> >>> I get the following error but the problem is more of a logical nature.
> >>> ./gnomeAppletMenu.py:40: GtkWarning: gtk_menu_shell_insert: assertion
> >>> `GTK_IS_MENU_ITEM (child)' failed
> >>> menu_bar.append(menu)
> >>> I'm pretty sure I fuck up around line 27-30
> >>> xml file:
> >>>http://pastie.org/480045
> >>> python code:
> >>>http://pastie.org/480042
> >> The traceback suggests to me that you can add only a 'MenuItem' to a
> >> 'MenuBar', but create_menu() is returning a 'Menu'.
>
> > Doesn't a MenuBar consists of Menu's?
>
> I've had a look at some examples on the web and it looks like you need
> to:
>
> 1. Set the menu on a menu item (the 'set_submenu' method);
> 2. Append the menu item onto the menu bar.
>
> What you're actually doing is:
>
> 1. Setting the menu on a menu item;
> 2. Appending the menu item onto a menu;
> 3. Trying to append the menu onto the menu bar.
>
> Result: exception.
>
> HTH
Ok, now it looks like below. I don't get any errors but the dictionary
entry show up as a toplevel menu... damn!!
def create_menu(node):
menus = []
for child in node.childNodes:
if child.localName == "item":
menus.append(gtk.MenuItem(child.getAttribute("name")))
if child.localName == "seperator":
menus.append(gtk.SeparatorMenuItem())
if child.localName == "menu": #if child.childNodes:
menuitem = gtk.MenuItem(child.getAttribute("name"))
menu = gtk.Menu()
for mi in create_menu(child): # for each menuitem
menu.append(mi) # append each menuitem
to menu
menuitem.set_submenu(menu) # set menu as submenu
of menuitem
menus.append(menuitem)
return menus
def factory(applet, iid):
doc = minidom.parse("menu.xml")
rootNode = doc.documentElement
menu_bar = gtk.MenuBar()
for cn in rootNode.childNodes: # for each menu under menus
for menu in create_menu(cn):# for each menu in list
menu_bar.append(menu) # append each menu
applet.add(menu_bar)
applet.show_all()
return True
--
http://mail.python.org/mailman/listinfo/python-list
Re: xml menu pygtk
On May 16, 5:04 pm, MRAB wrote: > Seb wrote: > > On May 16, 4:20 pm, MRAB wrote: > >> Seb wrote: > >>> I'm trying to construct a menu from an xml file. However my recursive > >>> algorithm isn't doing what I want it too. I've been starring at this > >>> for too long. Any help appreciated :) > >>> I get the following error but the problem is more of a logical nature. > >>> ./gnomeAppletMenu.py:40: GtkWarning: gtk_menu_shell_insert: assertion > >>> `GTK_IS_MENU_ITEM (child)' failed > >>> menu_bar.append(menu) > >>> I'm pretty sure I fuck up around line 27-30 > >>> xml file: > >>>http://pastie.org/480045 > >>> python code: > >>>http://pastie.org/480042 > >> The traceback suggests to me that you can add only a 'MenuItem' to a > >> 'MenuBar', but create_menu() is returning a 'Menu'. > > > Doesn't a MenuBar consists of Menu's? > > I've had a look at some examples on the web and it looks like you need > to: > > 1. Set the menu on a menu item (the 'set_submenu' method); > 2. Append the menu item onto the menu bar. > > What you're actually doing is: > > 1. Setting the menu on a menu item; > 2. Appending the menu item onto a menu; > 3. Trying to append the menu onto the menu bar. > > Result: exception. > > HTH I managed to get it to work. I thanks a lot for your help. best regards, Seb -- http://mail.python.org/mailman/listinfo/python-list
flock trouble
I'm trying to implement a file server using the code below. However
the locking doesn't work. I can delete while put'ing a file.
Anyone got an idea about why?
best regards,
seb
#! /usr/bin/env python
import Pyro.core, Pyro.naming
from Pyro.errors import PyroError, NamingError
import sys
import urllib
import os
import fcntl
class fileServer(Pyro.core.ObjBase):
basePath = "/home/snot/diku/dist/opg2/files_to_serve"
def __init__(self):
Pyro.core.ObjBase.__init__(self)
self.basePath = self.basePath.strip("..")
if not os.path.isdir(self.basePath) or
os.path.islink(self.basePath):
raise "invalid path"
def get(self, uri):
f = open(self.basePath + uri, 'r+')
fcntl.flock(f, fcntl.LOCK_SH)
data = f.read()
fcntl.flock(f, fcntl.LOCK_UN)
f.close()
return data
def put(self, uri, payload):
f = open(self.basePath + urllib.unquote_plus(uri), 'w')
fcntl.flock(f, fcntl.LOCK_EX)
f.truncate()
f.write(payload)
fcntl.flock(f, fcntl.LOCK_UN)
f.close()
def delete(self, uri):
f = open(self.basePath + urllib.unquote_plus(uri), 'w')
fcntl.flock(f, fcntl.LOCK_EX)
os.unlink(self.basePath + uri)
fcntl.flock(f, fcntl.LOCK_UN)
f.close()
try:
Pyro.core.initServer()
daemon = Pyro.core.Daemon()
locator = Pyro.naming.NameServerLocator()
print 'Searching for Name Server...'
try:
ns = locator.getNS()
except Pyro.errors.PyroError, message:
print message
sys.exit(1)
daemon.useNameServer(ns)
try:
ns.unregister("fileServer")
except NamingError:
pass
uri = daemon.connect(fileServer(), "fileServer")
print "The daemon runs on port:", daemon.port
print "The object's uri is:", uri
daemon.requestLoop()
except KeyboardInterrupt:
ns.unregister("fileServer")
print "ctrl + c pressed"
--
http://mail.python.org/mailman/listinfo/python-list
ssl server
I'm making a ssl server, but I'm not sure how I can verify the
clients. What do I actually need to place in _verify to actually
verify that the client cert is signed by me?
50 class SSLTCPServer(TCPServer):
51 keyFile = "sslcert/server.key"
52 certFile = "sslcert/server.crt"
53 def __init__(self, server_address, RequestHandlerClass):
54 ctx = SSL.Context(SSL.SSLv23_METHOD)
55 ctx.use_privatekey_file(self.keyFile)
56 ctx.use_certificate_file(self.certFile)
57 ctx.set_verify(SSL.VERIFY_PEER |
SSL.VERIFY_FAIL_IF_NO_PEER_CERT | SSL.VERIFY_CLIENT_ONCE,
self._verify)
58 ctx.set_verify_depth(10)
59 ctx.set_session_id('DFS')
60
61 self.server_address = server_address
62 self.RequestHandlerClass = RequestHandlerClass
63 self.socket = socket.socket(self.address_family,
self.socket_type)
64 self.socket = SSL.Connection(ctx, self.socket)
65 self.socket.bind(self.server_address)
66 self.socket.listen(self.request_queue_size)
67
68 def _verify(self, conn, cert, errno, depth, retcode):
69 return not cert.has_expired() and
cert.get_issuer().organizationName == 'DFS'
--
http://mail.python.org/mailman/listinfo/python-list
Re: ssl server
On Sep 17, 10:53 pm, "Giampaolo Rodola'" <[EMAIL PROTECTED]> wrote:
> On 17 Set, 19:33, Seb <[EMAIL PROTECTED]> wrote:
>
>
>
> > I'm making a ssl server, but I'm not sure how I can verify the
> > clients. What do I actually need to place in _verify to actually
> > verify that the client cert is signed by me?
>
> > 50 class SSLTCPServer(TCPServer):
> > 51 keyFile = "sslcert/server.key"
> > 52 certFile = "sslcert/server.crt"
> > 53 def __init__(self, server_address, RequestHandlerClass):
> > 54 ctx = SSL.Context(SSL.SSLv23_METHOD)
> > 55 ctx.use_privatekey_file(self.keyFile)
> > 56 ctx.use_certificate_file(self.certFile)
> > 57 ctx.set_verify(SSL.VERIFY_PEER |
> > SSL.VERIFY_FAIL_IF_NO_PEER_CERT | SSL.VERIFY_CLIENT_ONCE,
> > self._verify)
> > 58 ctx.set_verify_depth(10)
> > 59 ctx.set_session_id('DFS')
> > 60
> > 61 self.server_address = server_address
> > 62 self.RequestHandlerClass = RequestHandlerClass
> > 63 self.socket = socket.socket(self.address_family,
> > self.socket_type)
> > 64 self.socket = SSL.Connection(ctx, self.socket)
> > 65 self.socket.bind(self.server_address)
> > 66 self.socket.listen(self.request_queue_size)
> > 67
> > 68 def _verify(self, conn, cert, errno, depth, retcode):
> > 69 return not cert.has_expired() and
> > cert.get_issuer().organizationName == 'DFS'
>
> What library are you using? PyOpenSSL?
> In that case I think you'll have more luck by posting on their mailing
> list.
Thanks, I did that and it worked.
--
http://mail.python.org/mailman/listinfo/python-list
Re: ssl server
On Sep 18, 1:05 am, Michael Palmer <[EMAIL PROTECTED]> wrote:
> On Sep 17, 1:33 pm, Seb <[EMAIL PROTECTED]> wrote:
>
>
>
> > I'm making a ssl server, but I'm not sure how I can verify the
> > clients. What do I actually need to place in _verify to actually
> > verify that the client cert is signed by me?
>
> > 50 class SSLTCPServer(TCPServer):
> > 51 keyFile = "sslcert/server.key"
> > 52 certFile = "sslcert/server.crt"
> > 53 def __init__(self, server_address, RequestHandlerClass):
> > 54 ctx = SSL.Context(SSL.SSLv23_METHOD)
> > 55 ctx.use_privatekey_file(self.keyFile)
> > 56 ctx.use_certificate_file(self.certFile)
> > 57 ctx.set_verify(SSL.VERIFY_PEER |
> > SSL.VERIFY_FAIL_IF_NO_PEER_CERT | SSL.VERIFY_CLIENT_ONCE,
> > self._verify)
> > 58 ctx.set_verify_depth(10)
> > 59 ctx.set_session_id('DFS')
> > 60
> > 61 self.server_address = server_address
> > 62 self.RequestHandlerClass = RequestHandlerClass
> > 63 self.socket = socket.socket(self.address_family,
> > self.socket_type)
> > 64 self.socket = SSL.Connection(ctx, self.socket)
> > 65 self.socket.bind(self.server_address)
> > 66 self.socket.listen(self.request_queue_size)
> > 67
> > 68 def _verify(self, conn, cert, errno, depth, retcode):
> > 69 return not cert.has_expired() and
> > cert.get_issuer().organizationName == 'DFS'
>
> If I were you, I would just just hide behind apache, nginx oder
> another server that does ssl. just have that server proxy locally to
> your python server over http, and firewall the python server port.
Good idea, however atm this is a school project so thats not really an
option right now. However I might take this a bit furtherer and use
that solution.
--
http://mail.python.org/mailman/listinfo/python-list
Re: ssl server
On Sep 17, 7:33 pm, Seb <[EMAIL PROTECTED]> wrote:
> I'm making a ssl server, but I'm not sure how I can verify the
> clients. What do I actually need to place in _verify to actually
> verify that the client cert is signed by me?
>
> 50 class SSLTCPServer(TCPServer):
> 51 keyFile = "sslcert/server.key"
> 52 certFile = "sslcert/server.crt"
> 53 def __init__(self, server_address, RequestHandlerClass):
> 54 ctx = SSL.Context(SSL.SSLv23_METHOD)
> 55 ctx.use_privatekey_file(self.keyFile)
> 56 ctx.use_certificate_file(self.certFile)
> 57 ctx.set_verify(SSL.VERIFY_PEER |
> SSL.VERIFY_FAIL_IF_NO_PEER_CERT | SSL.VERIFY_CLIENT_ONCE,
> self._verify)
> 58 ctx.set_verify_depth(10)
> 59 ctx.set_session_id('DFS')
> 60
> 61 self.server_address = server_address
> 62 self.RequestHandlerClass = RequestHandlerClass
> 63 self.socket = socket.socket(self.address_family,
> self.socket_type)
> 64 self.socket = SSL.Connection(ctx, self.socket)
> 65 self.socket.bind(self.server_address)
> 66 self.socket.listen(self.request_queue_size)
> 67
> 68 def _verify(self, conn, cert, errno, depth, retcode):
> 69 return not cert.has_expired() and
> cert.get_issuer().organizationName == 'DFS'
Simply return retcode and it will work... assuming you have the certs
setup properly.
--
http://mail.python.org/mailman/listinfo/python-list
Re: multicast
Forgot the code... doh! :) #! /usr/bin/env python import socket import time class MulticastSender(object): def __init__(self, MCAST_ADDR = "224.168.2.9", MCAST_PORT = 1600): self.MCAST_ADDR = MCAST_ADDR self.MCAST_PORT = MCAST_PORT ANY = "0.0.0.0" SENDERPORT=1501 #create a UDP socket self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP) #allow multiple sockets to use the same PORT number self.sock.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1) #The sender is bound on (0.0.0.0:1501) self.sock.bind((ANY,SENDERPORT)) #Tell the kernel that we want to multicast and that the data is sent #to everyone (255 is the level of multicasting) self.sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 255) def send(self, data): self.sock.sendto(data, (self.MCAST_ADDR, self.MCAST_PORT)); class MulticastReceiver(object): def __init__(self, MCAST_ADDR = "224.168.2.9", MCAST_PORT = 1600): ANY = "0.0.0.0" #create a UDP socket self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP) #allow multiple sockets to use the same PORT number self.sock.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1) #Bind to the port that we know will receive multicast data self.sock.bind((ANY,MCAST_PORT)) #tell the kernel that we are a multicast socket self.sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 255) #Tell the kernel that we want to add ourselves to a multicast group #The address for the multicast group is the third param status = self.sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, socket.inet_aton(MCAST_ADDR) + socket.inet_aton(ANY)); self.sock.setblocking(0) def setblocking(self, flag): self.sock.setblocking(flag) def recv(self, size = 1024): return self.sock.recvfrom(size) class Multicast(object): def __init__(self): self.__ms = MulticastSender() self.__mr = MulticastReceiver() def send(self, data): self.__ms.send(data) def recv(self, size = 1024): return self.__mr.recv() if __name__ == "__main__": mc = Multicast() while 1: try: data, addr = mc.recv() except socket.error, e: #print "sock.error: ", e pass else: print "FROM: ", addr print "DATA: ", data -- http://mail.python.org/mailman/listinfo/python-list
conditional for-statement
Hi, i was wondering if there is a syntax alike: for i in range(10) if i > 5: print i equivalent to for i in (for i in range(10) if i>5): print i sebastien -- http://mail.python.org/mailman/listinfo/python-list
Re: conditional for-statement
On Aug 23, 6:18 pm, John Posner wrote: > >> Hi, > > >> i was wondering if there is a syntax alike: > > >> for i in range(10) if i > 5: > >> print i > > > You can write > > > for i in filter(lambda i: i > 5, range(10)): > > print i > > > but > > > for i in range(10): > > if i > 5: > > print i > > > it' better readable, and > > > for i in range(6,10): > > print i > > > it's event better. > > How about using a generator expression instead of a list? > > for i in (x for x in range(10) if x > 5): > print i > > -John Indeed, but we could have the same syntax than for generators but directly in the for statement as in for variable in generator if condition: body Is there a special reason for not doing so ? A rejected PEP ? -- http://mail.python.org/mailman/listinfo/python-list
Re: conditional for-statement
On Aug 23, 10:36 pm, seb wrote: > On Aug 23, 6:18 pm, John Posner wrote: > > > > > > > >> Hi, > > > >> i was wondering if there is a syntax alike: > > > >> for i in range(10) if i > 5: > > >> print i > > > > You can write > > > > for i in filter(lambda i: i > 5, range(10)): > > > print i > > > > but > > > > for i in range(10): > > > if i > 5: > > > print i > > > > it' better readable, and > > > > for i in range(6,10): > > > print i > > > > it's event better. > > > How about using a generator expression instead of a list? > > > for i in (x for x in range(10) if x > 5): > > print i > > > -John > > Indeed, but we could have the same syntax than for generators but > directly in the for statement as in > for variable in generator if condition: > body > > Is there a special reason for not doing so ? A rejected PEP ?- Hide quoted > text - > > - Show quoted text - I just found the same thread on the python ideas group at http://mail.python.org/pipermail/python-ideas/2009-April/004278.html sorry for reposting -- http://mail.python.org/mailman/listinfo/python-list
Re: conditional for-statement
On Aug 23, 11:02 pm, Chris Rebert wrote:
> On Sun, Aug 23, 2009 at 1:36 PM, seb wrote:
> > On Aug 23, 6:18 pm, John Posner wrote:
> >> >> Hi,
>
> >> >> i was wondering if there is a syntax alike:
>
> >> >> for i in range(10) if i > 5:
> >> >> print i
>
> >> > You can write
>
> >> > for i in filter(lambda i: i > 5, range(10)):
> >> > print i
>
> >> > but
>
> >> > for i in range(10):
> >> > if i > 5:
> >> > print i
>
> >> > it' better readable, and
>
> >> > for i in range(6,10):
> >> > print i
>
> >> > it's event better.
>
> >> How about using a generator expression instead of a list?
>
> >> for i in (x for x in range(10) if x > 5):
> >> print i
>
> >> -John
>
> > Indeed, but we could have the same syntax than for generators but
> > directly in the for statement as in
> > for variable in generator if condition:
> > body
>
> > Is there a special reason for not doing so ? A rejected PEP ?
>
> It's not been added since it's completely unnecessary (see the several
> alternatives already presented by others).
> There have been a few other mailinglist threads on adding essentially
> the same syntax. None have proved fruitful.
>
> Cheers,
> Chris
> --http://blog.rebertia.com- Hide quoted text -
>
> - Show quoted text -
On Aug 23, 11:02 pm, Chris Rebert wrote:
> On Sun, Aug 23, 2009 at 1:36 PM, seb wrote:
> > On Aug 23, 6:18 pm, John Posner wrote:
> >> >> Hi,
>
> >> >> i was wondering if there is a syntax alike:
>
> >> >> for i in range(10) if i > 5:
> >> >> print i
>
> >> > You can write
>
> >> > for i in filter(lambda i: i > 5, range(10)):
> >> > print i
>
> >> > but
>
> >> > for i in range(10):
> >> > if i > 5:
> >> > print i
>
> >> > it' better readable, and
>
> >> > for i in range(6,10):
> >> > print i
>
> >> > it's event better.
>
> >> How about using a generator expression instead of a list?
>
> >> for i in (x for x in range(10) if x > 5):
> >> print i
>
> >> -John
>
> > Indeed, but we could have the same syntax than for generators but
> > directly in the for statement as in
> > for variable in generator if condition:
> >body
>
> > Is there a special reason for not doing so ? A rejected PEP ?
>
> It's not been added since it's completely unnecessary (see the several
> alternatives already presented by others).
> There have been a few other mailinglist threads on adding essentially
> the same syntax. None have proved fruitful.
>
> Cheers,
> Chris
> --http://blog.rebertia.com- Hide quoted text -
>
> - Show quoted text -
Tx Chris for your reply !
i am still a bit puzzle by the following.
I read in http://en.wikipedia.org/wiki/Python_syntax_and_semantics#Generators
"""Python 3.0 unifies all collection types by introducing dict and set
comprehensions, similar to list comprehensions:
>>> [ n*n for n in range(5) ] # regular list comprehension
[0, 1, 4, 9, 16]
>>>
>>> { n*n for n in range(5) } # set comprehension
{0, 1, 4, 16, 9}
>>>
>>> { n: n*n for n in range(5) } # dict comprehension
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
"""
and we can add to this list the quite similar syntax for generator
expressions.
On all these loop constructs, one can consistenly add filtering on a
condition by adding an "if ..." after the "for ... in ..." part (and
it looks to me difficult to argue, for instance, that we should not
allow filtering for dict comprehesion because we could get the same
result by some other construct)
If the "if ..." part after the "for ... in ..." is so much used in all
these list/dict/set comprehensions and in the generator expressions,
it makes sense to me to have it also for the "for as a statement"
syntax :
[ n*n for n in range(10) if n%3 == 0]
{ n*n for n in range(10) if n%3 == 0}
{ n: n*n for n in range(10) if n%3 == 0}
( n*n for n in range(10) if n%3 == 0)
for n in range(10) if n%3 == 0:
print n*n
In fact, we often see the list comprehension [ n*n for n in range(10)
if n%3 == 0] explained as being equivalent to
l = []
for n in range(10):
if n%3 == 0:
l.append(n)
We could as consistenly explain that the syntax
for n in range(10) if n%3==0:
body
means
for n in range(10):
if n%3==0:
body
This syntax has also the benefit of avoiding an extra level of
indentation (the one for the if) that bears no real meaning on a
structural level.
Maybe a PEP could do the job...
Sébastien
--
http://mail.python.org/mailman/listinfo/python-list
Re: conditional for-statement
On Aug 24, 12:05 am, Mel wrote: > seb wrote: > > On Aug 23, 6:18 pm, John Posner wrote: > [ ... ] > >> How about using a generator expression instead of a list? > > >> for i in (x for x in range(10) if x > 5): > >> print i > > >> -John > > > Indeed, but we could have the same syntax than for generators but > > directly in the for statement as in > > for variable in generator if condition: > > body > > > Is there a special reason for not doing so ? A rejected PEP ? > > Well, the Zen of Python does say > > There should be one-- and preferably only one --obvious way to do it. > > Beyond that, I refer you to Gerald M. Weinberg's _The Psychology of Computer > Programming_, specifically chapters 11 and 12, about Programming Languages, > and their design. > > The proposal creates an case where one particular pair of syntactic > constructs can be mooshed together. OK for them, but everything else > becomes an exception; what about > > while a==c if b != d: > > why not > > if b != d while a==c: > what would be the unambiguous meaning of any of these forms ? they could be interesting but I do not understand them (yet!). > or > > for a in range(7) if os.name == 'posix': > > It winds up burdening the programmers with remembering which constructs are > and which are not mooshable. Weinberg gave an example: FORTRAN had some > stringent rules for what expressions were and were not allowed as array > subscripts. The result was that many programmers couldn't remember all the > rules, and often avoided using legal forms, having forgotten they were > legal. indeed, the language features should stay as orthogonal as possible (at least as a general rule) > > Maybe the line was already crossed when list comprehensions came into being, > still, the damage is localized in a particular context: building a list. It > isn't out creating wild options in the program control flow at large. > indeed and it proved to be very useful & successful. one cannot recommend to mix all features together as a general rule but using common sense and practice, we can have some exceptions that are definitely worth. > Mel.- Hide quoted text - > > - Show quoted text - -- http://mail.python.org/mailman/listinfo/python-list
Re: conditional for-statement
On Aug 25, 9:42 pm, Falcolas wrote:
> On Aug 25, 11:25 am, seb wrote:
>
>
>
> > We could as consistenly explain that the syntax
>
> > for n in range(10) if n%3==0:
> > body
>
> > means
>
> > for n in range(10):
> > if n%3==0:
> > body
>
> > This syntax has also the benefit of avoiding an extra level of
> > indentation (the one for the if) that bears no real meaning on a
> > structural level.
>
> > Maybe a PEP could do the job...
>
> > Sébastien
>
> So, what part of the statement does the "if" statement belong to;
> particularly a concern considering this is valid python:
>
> for x in y if y else z:
> body
>
can this be done in list/set/dict comprehensions/generator
expressions ?
> You can always do the following at the cost of 6 symbols, and the gain
> of clarity:
>
> for n in (x for x in y if y%3==0):
> body
it is in fact precisely to avoid this sort of line:
for n in (x for x in y if x%3==0):
and have instead the (more readable IMO) line
for n in y if n%3==0:
with:
- 1 "for ... in ..." instead of 2 (where one is the repetition of the
other)
- no parentheses
- no extra technical variable with local binding to the expression
generator ('x')
it looks more pythonic to me but it is a personal taste.
>
> ~G
--
http://mail.python.org/mailman/listinfo/python-list
Re: conditional for-statement
On Aug 25, 10:46 pm, Falcolas wrote: > On Aug 25, 1:58 pm, seb wrote: > > > On Aug 25, 9:42 pm, Falcolas wrote: > > > On Aug 25, 11:25 am, seb wrote: > > > So, what part of the statement does the "if" statement belong to; > > > particularly a concern considering this is valid python: > > > > for x in y if y else z: > > > body > > > can this be done in list/set/dict comprehensions/generator > > expressions ? > > It's a statement, so anywhere you use a statement (such as in > generators and list comprehensions), it can exist. for ... in ... only > requires that the statement return an iterable, it doesn't matter what > statement you use to get there. > I never thought about this case... Testing that in python 3.0, i see that >>> [x for x in range(5) if False else range(10)] SyntaxError: invalid syntax (, line 1) >>> [x for x in (range(5) if False else range(10))] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] hence, to avoid the ambiguity you mentionned (that already exists with list comprehensions), one can use the parenthesis. This minor incompatibility is referenced in http://www.python.org/dev/peps/pep-0308/ > It doesn't feel clear to me, which is why I would have to disagree. > IIRC, these very filters are the main reason that list comprehensions, > and later one-line generators and dictionary comprehensions were > originally created: so you can filter the stream before acting on it. > One statement performs one action - very precise and understandable. in my mind, I am thinking as you are but with 'loop' instead of 'stream' """so you can filter the 'loop' before acting on it. One statement performs one action - very precise and understandable.""" and it doesn't look that dissimilar to your own reasoning. > > It's just my two cents, but I would not agree that adding a filter > keyword to for loops is required. Not when we have the filter > function, various types of comprehensions, and if statements which all > provide that very functionality. > before having list comprehensions, we could also have said that """ I would not agree that adding a 'list comprehension' feature is required. Not when we have the filter function, and the 'for+if' statements which all provide that very functionality.""" How can unifying the "for ... in ..." statement with the "for ... in ... if ..." syntax be detrimental ? It would be an inconsistence less to remember, wouldn't it ? cheers, sebastien -- http://mail.python.org/mailman/listinfo/python-list
Re: conditional for-statement
On Aug 25, 11:57 pm, Piet van Oostrum wrote: > You can also say: > [x+y for x in range(3) for y in range(4) if x < y] > If you want to write this as a loop you have to put the for's on > separate lines separated by colons, so why not the if also? Or would you > also like to have the for's on one line? indeed, we could have the for's on one line too. In fact, if you have a double loop without any body that is specific to the outer loop, you can merge the two lines. Similarly, if you have a loop with a if (ie filter) without any body having to be executed outside the if, having them on the same line is also clearer. | for x in range(3): | for x in range(3) for y in range(4): |for y in range(4): could be written as |body | body and | for x in range(3): | for x in range(3) if cond(x): |if cond(x): could be written as |body | body Such loops are in fact mentally just one loop over the cartesian product (product set) of the two iterators or over the filter set of the iterator respectively in these examples. Moreover, for 'body' longer than a couple of lines, it is quite useful to see immediately that there is no additional code that could be only specific to the outher loop ie, when writing all in one line, we can be sure that we do not have this situation | for x in range(3): |for y in range(4): | long body | ... | long body |body related to outer loop The naturalness of iteration in the various comprehensions and the generator expressions can be smoothly extended to the standard code iteration. BTW, I noticed this inconsistency when, coming back to python after a couple of weeks of no python programming, I started coding loops like for obj in list_obj if obj.Size > 100: print(obj) and got an error message about syntax. > -- > Piet van Oostrum > URL:http://pietvanoostrum.com[PGP 8DAE142BE17999C4] > Private email: p...@vanoostrum.org- Hide quoted text - > > - Show quoted text - -- http://mail.python.org/mailman/listinfo/python-list
http2https proxy
I'd like to open a ssl connection to a https server, done Create a socket and bind it to a local port, done Connect the two in such a way that everything read or written to the local port is actually read or written to the https server. In other words I want a http2https proxy. ideas? best regards, Seb -- http://mail.python.org/mailman/listinfo/python-list
Distutils beginner question - windows
Hi all, Just a quick question , I have a simple script I want to convert into a windows installer and give to some friends. I had a look at http://docs.python.org/distutils/introduction.html and wrote this setup script: #!/usr/bin/env python from distutils.core import setup setup(name="C:\data\Sendmailmsg.py", version='1.0', description='Python Distribution Utilities', author='Sebas929', author_email=' ', url=' ', py_modules=['urllib','smtplib'], ) I tried to run this - "C:\Data\Setup.py" bdist_wininst - in a cmd prompt. C:\Data\ contains my script Sendmailmsg.py and Setup.py I am getting the error : file urllib.py (for module urllib) not found file smtplib.py (for module smtplib) not found file urllib.py (for module urllib) not found file smtplib.py (for module smtplib) not found warning: install_lib: 'build\lib' does not exist -- no Python modules to install This creates an installer which crashes when I use it. I have a few questions: How can I fix this error ? Can I use '.\myscript.py' in the name parameter to make it look in the same directory as setup.py? When I have it as an installer what happens? When I install it will there be something you can click which will run the script? I have never tried this before so I am probably missing something obvious, thanks in advance for any help. -- http://mail.python.org/mailman/listinfo/python-list
why does php have a standard SQL module and Python doesn't !?
Hi, I thought I would quickly write a "SELECT * FROM ... " line in python ... ... but then I was suprised to find that the only "official" web page I found was mainly linking to the PEP on HOW the DB API2.0 should look like. Then there is a choice of maybe ten modules which more or less all claim to implement 99% of that PEP. What is the best (easiest ! quickest !) way to get started doing some SQL in Python ? (I have mySQL running on a Linux box in our Lab and would like to access it from Windows (or MacOSX) on some other machine also in the same subnet) Sebastian Haase UCSF -- http://mail.python.org/mailman/listinfo/python-list
Re: why does php have a standard SQL module and Python doesn't !?
This was my point though: I found the *description* - but no wordon WHICH implementation to get WHERE ? Don't get me wrong - I like Python and I will stick to it for some time to come ... but WHY are the places to get those SQL implementations not better marked: Just try to google for "python sql" and you get here: http://sourceforge.net/projects/pysqldb/ (then this like to home page (http://pysqldb.sourceforge.net/) is COMPLETELY EMPTY ) the next number of hits seem to be completely random. Even if you google for: python mysql you get to http://sourceforge.net/projects/mysql-python where the linked "home page" says: ** OBSOLETE Python Interface to MySQL ** Please try to understand what I'm saying here: This is essentially a proof the there is no (one) good place to go for SQL & Python. Instead it looked really scary to me! How could I have known that "MySQLdb" is the keyword to google for !? It would be nice, if those projects would reference back to "http://www.python.org/topics/database/modules.html"; just to give some assurance that one is not totally off ... Thanks, Sebastian -- http://mail.python.org/mailman/listinfo/python-list
Re: why does php have a standard SQL module and Python doesn't !?
OK, point taken - maybe what threw me off was the simple fact that there CAN be NO "ONE standard/default SQL package". As a newbie in sql I was hoping to find something like e.g. the socket module (one size fits all) So: Maybe this could be explained on the "Database Modules" page. (and again: just being on sourceforge doesn't mean a project is still being developped - for all one call tell from the first look, they could be abandoned - hint: have a better homepage !!) Thanks, Sebastian -- http://mail.python.org/mailman/listinfo/python-list
Re: OpenGL
Does that mean PyOpenGL is based on ctypes ? I thought is was using SWIG ? And on that note: I have some (old) SWIG typemaps for numarray arrays that I'm using for many years. I was always wondering how difficult it would be for me to add them into PyOpenGL ? So far I'm just using my own (one !) OpenGL function with that typemap to draw 6 vertices in a vector linestrip. But I would like to not have to resort to "string conversion" when for example drawing 2D typemaps... Anyway, thanks a lot for PytOpenGL - my program (image analysis platform and microscope control) would not be possible without it. Sebastian Haase -- http://mail.python.org/mailman/listinfo/python-list
how to create a movie ?
Hi, I have a series of images (either from numarray or from PIL) After googling if read something about pyMedia... Can I create an AVI movie from my images with that ? How about quicktime ? Thanks, Sebastian Haase -- http://mail.python.org/mailman/listinfo/python-list
Python 3000 deat !? Is true division ever coming ?
Hi, Is it true that that "Python 3000" is dead ? Honestly I think that e.g. changing 5/2 to be 2.5 (instead of 2) would just break to much code :-( On the otherhand I'm using Python as "Matlab replacement" and would generally like 5/2 ==2.5 So, I was contemplating to default all my modules/scripts to start with "from __future__ import division" but if it is never coming (in this decade, that is) then it would be a waste of time just confuse everybody !! Thanks, Sebastian Haase -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 3000 deat !? Is true division ever coming ?
Thanks for the replies, But to point out what the subject of this thread is (sorry for the typo ;-) : There is a PEP (proposal 238) to change Python so that 5/2 WOULD do the true division -- and obviously break lots of code. Just type this in your python interpeter: >>> from __future__ import division >>> 5/2 2.5 I'm not asking why it _doen't_ work like Matlab. I'm wondering what the chances are that what Rocco Moretti is referring to (http://www.python.org/doc/2.2.3/whatsnew/node7.html) is really planned to be persued... I don't think "Because this change might break code, it's being introduced very gradually. Python 2.2 begins the transition, but the switch won't be complete until Python 3.0." says enough. Should I start using "from __future__ import division" in my modules ? I fear that it might make my code "unreadable" (people would not expect this when reading somewhere in the middle of the module !!!) Thanks, Sebastian Haase -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 3000 deat !? Is true division ever coming ?
Thank you very much, Magnus ! This is the answer I had been waiting for: > A problem as I see it today, is that this behaviour is > not actively encouraged. The tutorial, which is maintained > and updated, still describes old style classes, and the > old division behaviour. My main point was/is: why is there not more discussion about "true division" !!? Just like the second answer to my posting clearly showed: PEOPLE THINK TRUE DIVISION IS "ONLY IN MATLAB" !! As you pointed out: the "true division" part of "Python3000" might be one of the "scariest" and should therefore be pointed out already in the tutorial !! (It would look quite ugly to newcomers, though) Having said that: I would vote against EVER introducing true division as default - because it will just PISS too many (long time python) people OFF. ;-) Thanks, Sebastian Haase Magnus Lycka wrote: > [EMAIL PROTECTED] wrote: > > Hi, > > Is it true that that "Python 3000" is dead ? > > I think you should view Python 3000 as a metaphor for > "Python as it would look if we didn't have to care about > backward compatibility". > > Before this name appeared, Guido used to talk about > Python 3.0 as a version where bad things in Python could > go away, but it seems many people got worried about that, > assuming that Python 3.0 would happen soon and cause a lot > of changes in already written code. Thus the less scary > label Python 3000. There has never been any concrete plans > as far as I know to actually release a real software product > called Python 3000. It's rather a long term design target > for Python. > > A lot of these language changes can be introduced step > be step. A new feature can be introduced first as an > experimental feature, accessible via "from __future__ > import XYZ", in the next minor release, it might be > enabled by default. If a feature or module is going away, > the first step is to document it as deprecated. The next > minor release will issue a PendingDeprecationWarning, and > the minor release after that will issue a DeprecationWarning > and the next minor release (2.4 -> 2.5 etc) will remove > the feature. This will give developers a period of many > years to adapt from the time the feature is documented as > deprecated until maintenace ends for the last Python where > the deprecated feature works. > > For an example of such a transition plan, see > http://www.python.org/peps/pep-0352.html > > Changing the behaviour of int/int is worse, since there > is no obvious way for Python to determine whether the > programmer expects an int result, a float result or either. > Also, the typical consequence of this change is not that > code breaks with a big bang, but rather that it produces > somewhat different results. That means that a bug due to > this feature change might go unnoticed for a long time, > and cause a lot of problems. Imagine some kind of system > that calculates how much pension you'll get when you > retire, based on your monthly payments. A bug caused by > changed division behaviour might have serious consequences. > > It seems reasonable to wait until the next major release, > 3.0, before this feature is introduced. In the mean time > it seems that the strategy should be to encourage people > to adopt the new behaviour in all new code, i.e. to use > "from __future__ import division" and to explicitly use > interger division a//b when this is needed. > > In the same way, people should be encouraged to always > use new style classes, "class X(object):" in new code, > since old style classes are going away, and they behave > differently in some ways. > > A problem as I see it today, is that this behaviour is > not actively encouraged. The tutorial, which is maintained > and updated, still describes old style classes, and the > old division behaviour. > > http://docs.python.org/dev/tut/node5.html#SECTION00511 > http://docs.python.org/dev/tut/node11.html#SECTION001130 > > I don't see any reason to use old style classes in new code, > so I think all "class X:" should be changed to > "class X(object):", but I can understand that it would be > confusing to explain the use of "from __future__ import division" > before we introduce simple arithmetics. Still, we should > get this message through! > > As it is now, when people aren't actively moving their code > towards this expected change, the impact of such a change > would be almost as strong as if this "from __future_..." > feature didn't exist. > > So, if I finally try to answer your question: Float division > will hardly be enabled by default until most Python programmers > have adopted the feature, i.e. enabled it through the > __future__ switch and started to use // when they want floor > division. > > This will hardly happen until it's documented as the way people > should do it. Perhaps a start could be a FutureProofPython page > in the Python wiki, with suggested coding guidelines. -- http://mail.python.org/mailman/listinfo
PyOpenGL without SetupTools
Hi, I am distributing a package with a precompiled collection of modules and packages useful for Python based medical/biological/astronomical image analysis and algorithm development. (Codename: Priithon). For Priithon I put all modules/packages in a simple / single directory (tree) including one starting batch/script file. This script sets up PYTHONPATH to find modules at this place. It works fine for Windows,Linux and Mac-OSX. Now I want to upgrade everything to Python 2.5 and thought it might be time get PyOpengl version 3 (aka. pyOpengl-ctypes). The problem at hand is now that PyOpenGL uses "all this setup-tools machinery" just to initialize the formathandlers for the different ways to deal with arrays. (I really need only numpy support !) This is done via the pkg_resources mechanism called "entry_points". Can I include a simple non-system-install of pkg_resources that makes at least the entry_point stuff work ? Where do I put pyOpenGL's "entry_points.txt" file ? Thanks, Sebastian Haase MDC Berlin -- http://mail.python.org/mailman/listinfo/python-list
Re: PyOpenGL without SetupTools
Are there PyOpenGL 2.0 (I guess 2.0.1.09 is goood) binaries available
for Python 2.5 ? Anywhere ?
Thanks for the reply
-Sebastian Haase
On Oct 1, 11:49 am, Carl Banks <[EMAIL PROTECTED]> wrote:
> On Oct 1, 4:04 am, [EMAIL PROTECTED] wrote:
>
>
>
> > Hi,
> > I am distributing a package with a precompiled collection of modules
> > and packages useful for Python based medical/biological/astronomical
> > image analysis and algorithm development. (Codename: Priithon).
> > For Priithon I put all modules/packages in a simple / single directory
> > (tree) including one starting batch/script file. This script sets up
> > PYTHONPATH to find modules at this place.
> > It works fine for Windows,Linux and Mac-OSX.
>
> > Now I want to upgrade everything to Python 2.5 and thought it might
> > be time get PyOpengl version 3 (aka. pyOpengl-ctypes).
>
> > The problem at hand is now that PyOpenGL uses "all this setup-tools
> > machinery" just to initialize the formathandlers for the different
> > ways to deal with arrays. (I really need only numpy support !)
> > This is done via the pkg_resources mechanism called "entry_points".
>
> > Can I include a simple non-system-install of pkg_resources that makes
> > at least the entry_point stuff work ? Where do I put pyOpenGL's
> > "entry_points.txt" file ?
>
> The simple answer is "don't bother with PyOpenGL-ctypes if you don't
> have to". Besides the hassles with binary packaging, it's quite a bit
> slower then PyOpenGL 2.0.
>
> Anyways, when I saw that I couldn't easily figure out how to register
> a setuptools package by hand (the point you seem to be at now), I
> resorted to using the same hack that setuptools uses to register its
> packages. Setuptools puts a pth file, something like
> "ezsetup_.pth", in the site-packages directory. The file has a
> couple lines of Python code that mark the start and end of the eggs on
> sys.path; then they call setuptools to postprocess and register the
> eggs.
>
> What I did was to duplicate the effect of these lines in my main
> script. I don't have it in front of me, but it was something along
> these lines:
>
> sys.__eggindex = len(sys.path)
> sys.path.append("OpenGL-cytpes-3.0.0a7-whatever.egg")
> setuptools.register_packages()
>
> I copied the OpenGL egg to my distro directory, and it worked.
>
> Another possibility is to do it the "setuptools" way: by not packaging
> PyOpenGL 3.0 at all, and instead marking it as a dependency. Then,
> setuptools will helpfully download it for the user. (Personally, I
> hate that scripts can recklessly download and install stuff into your
> site packages without even asking; seems like a possible security hole
> as well.) But it's an option.
>
> Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
How popular is Django ?
Hi! I was surprised when I did a google-groups search for python, ( http://groups.google.com/groups/search?q=python&qt_s=Search+Groups ) it shows these groups: comp.lang.python with about 11000 users, and second, Django users Discussion group for Django users. Django is a high-level Python Web ... with 6500 members I cannot believe that django is *that* popular ... where am I going wrong ? Cheers, Sebastian Haase -- http://mail.python.org/mailman/listinfo/python-list
where is pkg_resources (setuptools) looking for entry_points.txt
Hi, I'm trying to use an egg packages (pyOpenGL) as a "normal package" (by unzipping the egg into a directory). Generally this seems to work fine, except one init-step is missing. >From setuptools I got the pkg_resources.py file. But now the package is not properly initialized, because iter_entry_points() in pkg_resources does not find the corresponding entry_points.txt file needed to setup some intial configuration... (array handlers for pyOpenGL) Is there a default directory name or an environment variably or maybe the (ominous? ) __main__.__require variable that I can use to have the entry_points text read ? (I would even be willing to "manually" load the entry_points.txt file and feed it into some pkg_resources related variable to make it happy [before I import my package] ) Thanks, Sebastian Haase -- http://mail.python.org/mailman/listinfo/python-list
Re: Will multithreading make python less popular?
hi there, [snip] > Google uses Processes for the Tabs in Chrome, because that way they get > around many Memory Management Problems they would have with Threads or with > a singlethreaded reactor. Using Processes is not per se a bad Idea. You pay > a bit with Memory and CPU but in many situations you get a much simpler > programming model. note also that one can bet on things like KSM [1] to further improve the memory situation. For example, we are investigating its use in the large (read heavy) application frameworks at CERN where the typical vmem footprint is ~2Gb. Running KSM together with a multiprocessing-based event loop manager, we managed to overcommit ~300% of the memory (ie: run 3 instances of our application on a 2Gb-per-core machine) cheers, sebastien [1] http://lwn.net/Articles/306642 -- http://mail.python.org/mailman/listinfo/python-list
pickle: 32bit vs 64bit
Hi, I have ordered a new Athlon64 PC running linux x86_64. I'm using Pyro to communicate between python programs running on different machines. I know that, because Pyro's communication is based on pickle, that the different machines have to run the same Python (major) version: that is, it did not work after I just upgraded ONE machine from Py2.2 to Py2.4. How will this be when all machines run Python2.4, but one will be compiled as a 64bit application, the others 32bit ? (If it works for 2.4, will it still work for Python2.5, where lists can be larger than 2GB ONLY on 64bit-Python ? [of course I mean the case where the lists are actually (much) shorter than 2GB !!] ) Thanks, Sebastian Haase -- http://mail.python.org/mailman/listinfo/python-list
