Weighted "random" selection from list of lists

2005-10-08 Thread Jesse Noller
Hello - I'm probably missing something here, but I have a problem where I am populating a list of lists like this: list1 = [ 'a', 'b', 'c' ] list2 = [ 'dog', 'cat', 'panda' ] list3 = [ 'blue', 'red', 'green' ] main_list = [ list1, list2, list3 ] Once main_list is populated, I want to build a se

OT: Boston-Area QA/Python Job

2005-06-24 Thread Jesse Noller
Sorry for the off-topic post everyone. The company I work for has a job opening for a Senior QA/Automation person, and they are looking for someone strong in Python to help develop tests/testing frameworks/etc. The complete job description follows - you can feel free to email resumes and questions

Re: How can I import a py script by its absolute path name?

2005-07-14 Thread Jesse Noller
A question in a similiar vein: I have appended 2 different directories to my path (via sys.path.append) now - without knowing the names of the files in those directories, I want to force an import of the libraries ala: for f in os.listdir(os.path.abspath(libdir)): module_name = f.strip('.py')

Re: How to realize ssh & scp by Python

2005-07-24 Thread Jesse Noller
On 7/23/05, 刚 王 <[EMAIL PROTECTED]> wrote: > I would like to write a Python code like this: > > It can login a host by SSH > after login the host, use SCP to get a remote file, so it can deliver file > to the host. > then execute the program > then leave the host > > For example : > > STEP 1. ss

NDMP Library?

2005-09-09 Thread Jesse Noller
Does anyone know of a python module/library for communicating with the NDMP (ndmp.org) protocol? I'm doing some research and anything would be a great help, thanks! -jesse -- http://mail.python.org/mailman/listinfo/python-list

httplib and large file uploads

2006-10-02 Thread Jesse Noller
Hey All,I'm working on an script that will generate a file of N size (where N is 1k-1gig) in small chunks, in memory (and hash the data on the fly) and pass it to an httplib object for upload. I don't want to store the file on the disk, or completely in memory at any time. The problem arises after

Simple SMTP server

2005-06-09 Thread Jesse Noller
Hello - I am looking at implementing a simple SMTP server in python - I know about the smtpd module, but I am looking for code examples/snippets as the documentation is sparse. The server I am looking at writing is really simple - it just needs to fork/thread appropriately for handling large batc

Re: multiprocessing vs. distributed processing

2009-01-16 Thread Jesse Noller
On Fri, Jan 16, 2009 at 12:52 AM, James Mills wrote: > I've noticed over the past few weeks lots of questions > asked about multi-processing (including myself). > > For those of you new to multi-processing, perhaps this > thread may help you. Some things I want to start off > with to point out are

Re: process/thread instances and attributes

2009-01-16 Thread Jesse Noller
On Thu, Jan 15, 2009 at 10:37 PM, James Mills wrote: > After some work ... I've taken Laszlo's suggestion of using Value > (shared memory) objects > to share state between the -pseudo- Process (manager) object and it's > underlying multiprocessing.Process > instance (and subsequent process): > > H

Re: Python 2.6's multiprocessing lock not working on second use?

2009-01-19 Thread Jesse Noller
On Mon, Jan 19, 2009 at 8:16 AM, Frédéric Sagnes wrote: > On Jan 19, 11:53 am, Frédéric Sagnes wrote: >> On Jan 17, 11:32 am, "Gabriel Genellina" >> wrote: >> >> >> >> > En Fri, 16 Jan 2009 14:41:21 -0200, escribiste en el grupo >> > gmane.comp.python.general >> >> > > I ran a few tests on the n

Re: Python 2.6's multiprocessing lock not working on second use?

2009-01-19 Thread Jesse Noller
On Mon, Jan 19, 2009 at 1:32 PM, Nick Craig-Wood wrote: > Jesse Noller wrote: >> > Opened issue #4999 [http://bugs.python.org/issue4999] on the matter, >> > referencing this thread. >> >> Thanks, I've assigned it to myself. Hopefully I can get a fix put

Re: malloc (error code=12)

2009-01-21 Thread Jesse Noller
On Wed, Jan 21, 2009 at 9:38 PM, Arash Arfaee wrote: > > Hi All, > > I am writing a multiprocessing program using python 2.6. It works in most > cases, however when my input is large sometimes I get this message again and > again: > > Python(15492,0xb0103000) malloc: *** mmap(size=393216) failed (

Re: Addition of multiprocessing ill-advised? (was: Python 3.0.1)

2009-01-28 Thread Jesse Noller
On Tue, Jan 27, 2009 at 11:36 PM, James Mills wrote: > On Wed, Jan 28, 2009 at 1:49 PM, Ben Finney wrote: >> Steve Holden writes: >>> I think that [Python 2.6 was a rushed release]. 2.6 showed it in the >>> inclusion (later recognizable as somewhat ill-advised so late in the >>> day) of multipro

Re: Addition of multiprocessing ill-advised? (was: Python 3.0.1)

2009-01-28 Thread Jesse Noller
On Tue, Jan 27, 2009 at 10:49 PM, Ben Finney wrote: > (Continuing a side topic of a different discussion) > > Steve Holden writes: > >> I think that [Python 2.6 was a rushed release]. 2.6 showed it in the >> inclusion (later recognizable as somewhat ill-advised so late in the >> day) of multiproc

Re: Addition of multiprocessing ill-advised?

2009-01-28 Thread Jesse Noller
On Wed, Jan 28, 2009 at 8:32 AM, Steve Holden wrote: ...snip... >> I have found no problems with it - I've recently integrated it with my >> event/component framework (1). In my library I use Process, Pipe >> and Value. >> > It will be a great library in time, but the code was immature and > insuf

Re: Terminating a Python program that uses multi-process, multi-threading

2009-01-29 Thread Jesse Noller
On Wed, Jan 28, 2009 at 3:46 PM, akineko wrote: > Hello Python experts, > > I have a program that uses three processes (invoked by > multiprocessing) and several threads. > The program is terminated when ^C is typed (KeyboardInterrupt). > The main process takes the KeyboardInterrupt Exception and

Re: Where to host a (Python) project?

2009-01-31 Thread Jesse Noller
On Sat, Jan 31, 2009 at 4:30 PM, andrew cooke wrote: > On Jan 31, 4:50 pm, "Giampaolo Rodola'" wrote: >> Google Code. >> >> --- Giampaolohttp://code.google.com/p/pyftpdlib > > thanks - that's a nice example. i'm a bit concerned about the whole > google corporation thing, but reading through the

Re: PyYaml in standard library?

2009-02-18 Thread Jesse Noller
On Wed, Feb 18, 2009 at 4:11 AM, Brendan Miller wrote: > I'm just curious whether PyYaml is likely to end up in the standard > library at some point? > -- > http://mail.python.org/mailman/listinfo/python-list > Personally, loving the hell out of yaml (and pyyaml) as much as I do, I'd love to see

Re: multiprocessing module and os.close(sys.stdin.fileno())

2009-02-18 Thread Jesse Noller
On Tue, Feb 17, 2009 at 10:34 PM, Graham Dumpleton wrote: > Why is the multiprocessing module, ie., multiprocessing/process.py, in > _bootstrap() doing: > > os.close(sys.stdin.fileno()) > > rather than: > > sys.stdin.close() > > Technically it is feasible that stdin could have been replaced with

Re: Builing Python 2.6 on AIX 5.2

2008-10-06 Thread Jesse Noller
Looks like AIX is missing sem_timedwait - see: http://bugs.python.org/issue3876 Please add your error to the bug report just so I can track it. -jesse On Mon, Oct 6, 2008 at 4:16 AM, brasse <[EMAIL PROTECTED]> wrote: > Hello! > > I am having some trouble building Python 2.6 on AIX. The steps I h

Re: Builing Python 2.6 on AIX 5.2

2008-10-07 Thread Jesse Noller
Thanks for posting this to the tracker mattias - as soon as I can steal some time, I'll dig into it and see if I can get it teed up for the patch release. On Tue, Oct 7, 2008 at 6:24 AM, brasse <[EMAIL PROTECTED]> wrote: > On Oct 6, 10:16 am, brasse <[EMAIL PROTECTED]> wrote: >> Hello! >> >> I am

Re: Using multiprocessing

2008-10-10 Thread Jesse Noller
On Fri, Oct 10, 2008 at 4:32 PM, nhwarriors <[EMAIL PROTECTED]> wrote: > I am attempting to use the (new in 2.6) multiprocessing package to > process 2 items in a large queue of items simultaneously. I'd like to > be able to print to the screen the results of each item before > starting the next on

Re: Python 2.6, multiprocessing module and BSD

2008-10-22 Thread Jesse Noller
On Tue, Oct 21, 2008 at 6:45 PM, <[EMAIL PROTECTED]> wrote: > It seems that the multiprocessing module in 2.6 is broken for *BSD; > I've seen issue 3770 regarding this. I'm curious if there are more > details on this issue since the posts in 3770 were a bit unclear. For > example, one post claimed

Re: Python 2.6, multiprocessing module and BSD

2008-10-22 Thread Jesse Noller
On Wed, Oct 22, 2008 at 10:31 AM, <[EMAIL PROTECTED]> wrote: > On Oct 22, 8:11 am, "Jesse Noller" <[EMAIL PROTECTED]> wrote: >> On Tue, Oct 21, 2008 at 6:45 PM, <[EMAIL PROTECTED]> wrote: >> > It seems that the multiprocessing module in 2.6 is broken

Re: Python 2.6, multiprocessing module and BSD

2008-10-22 Thread Jesse Noller
On Wed, Oct 22, 2008 at 11:06 AM, Philip Semanchuk <[EMAIL PROTECTED]> wrote: >> The BSD issue was raised late in the cycle for 2.6. The problem is >> that FBSD's support is "very experimental" as Phillip points out - and >> OpenBSD doesn't even have them. >> >> Due to the lateness of the issue and

Re: 2.6, 3.0, and truly independent intepreters

2008-10-22 Thread Jesse Noller
On Wed, Oct 22, 2008 at 12:32 PM, Andy <[EMAIL PROTECTED]> wrote: > And, yes, I'm aware of the multiprocessing module added in 2.6, but > that stuff isn't lightweight and isn't suitable at all for many > environments (including ours). The bottom line is that if you want to > perform independent pr

Re: 2.6, 3.0, and truly independent intepreters

2008-10-22 Thread Jesse Noller
On Wed, Oct 22, 2008 at 5:34 PM, Terry Reedy <[EMAIL PROTECTED]> wrote: > The *current* developers seem to be more interested in exploiting multiple > processors with multiprocessing. Note that Google choose that route for > Chrome (as I understood their comic introduction). 2.6 and 3.0 come with

Re: 2.6, 3.0, and truly independent intepreters

2008-10-24 Thread Jesse Noller
On Fri, Oct 24, 2008 at 10:40 AM, Andy O'Meara <[EMAIL PROTECTED]> wrote: >> > 2) Barriers to "free threading". As Jesse describes, this is simply >> > just the GIL being in place, but of course it's there for a reason. >> > It's there because (1) doesn't hold and there was never any specs/ >> > g

Re: 2.6, 3.0, and truly independent intepreters

2008-10-24 Thread Jesse Noller
On Fri, Oct 24, 2008 at 12:30 PM, Jesse Noller <[EMAIL PROTECTED]> wrote: > On Fri, Oct 24, 2008 at 10:40 AM, Andy O'Meara <[EMAIL PROTECTED]> wrote: >>> > 2) Barriers to "free threading". As Jesse describes, this is simply >>> > just the GIL

Re: 2.6, 3.0, and truly independent intepreters

2008-10-24 Thread Jesse Noller
On Fri, Oct 24, 2008 at 3:17 PM, Andy O'Meara <[EMAIL PROTECTED]> wrote: > I'm a lousy writer sometimes, but I feel bad if you took the time to > describe threads vs processes. The only reason I raised IPC with my > "messaging isn't very attractive" comment was to respond to Glenn > Linderman's p

Re: 2.6, 3.0, and truly independent intepreters

2008-10-24 Thread Jesse Noller
On Fri, Oct 24, 2008 at 4:51 PM, Andy O'Meara <[EMAIL PROTECTED]> wrote: >> In the module multiprocessing environment could you not use shared >> memory, then, for the large shared data items? >> > > As I understand things, the multiprocessing puts stuff in a child > process (i.e. a separate addre

Re: 2.6, 3.0, and truly independent intepreters

2008-10-30 Thread Jesse Noller
On Wed, Oct 29, 2008 at 8:05 PM, Glenn Linderman <[EMAIL PROTECTED]> wrote: > On approximately 10/29/2008 3:45 PM, came the following characters from the > keyboard of Patrick Stinson: >> >> If you are dealing with "lots" of data like in video or sound editing, >> you would just keep the data in sh

Re: 2.6, 3.0, and truly independent intepreters

2008-10-30 Thread Jesse Noller
On Thu, Oct 30, 2008 at 12:05 PM, Andy O'Meara <[EMAIL PROTECTED]> wrote: > On Oct 28, 6:11 pm, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: >> > Because then we're back into the GIL not permitting threads efficient >> > core use on CPU bound scripts running on other threads (when they >> > otherwi

Re: 2.6, 3.0, and truly independent intepreters

2008-10-30 Thread Jesse Noller
On Thu, Oct 30, 2008 at 1:54 PM, Andy O'Meara <[EMAIL PROTECTED]> wrote: > On Oct 30, 1:00 pm, "Jesse Noller" <[EMAIL PROTECTED]> wrote: > >> >> Multiprocessing is written in C, so as for the "less agile" - I don't >> see how it

Re: [Python-Dev] PEP 382: Namespace Packages

2009-04-06 Thread Jesse Noller
On Thu, Apr 2, 2009 at 4:33 PM, M.-A. Lemburg wrote: > On 2009-04-02 17:32, Martin v. Löwis wrote: >> I propose the following PEP for inclusion to Python 3.1. > > Thanks for picking this up. > > I'd like to extend the proposal to Python 2.7 and later. > -1 to adding it to the 2.x series. There wa

Re: [Python-Dev] PEP 382: Namespace Packages

2009-04-06 Thread Jesse Noller
On Mon, Apr 6, 2009 at 9:26 AM, Barry Warsaw wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On Apr 6, 2009, at 9:21 AM, Jesse Noller wrote: > >> On Thu, Apr 2, 2009 at 4:33 PM, M.-A. Lemburg wrote: >>> >>> On 2009-04-02 17:32, Martin

Re: pyprocessing and exceptions

2009-04-15 Thread Jesse Noller
On Wed, Apr 15, 2009 at 11:32 AM, garyrob wrote: > Hi, > > We're still using Python 2.5 so this question is about the > pyprocessing module rather than the multiprocessing module, but I'm > guessing the answer is the same. > > I tend to use the Pool() object to create slave processes. If > somethi

Re: Ending data exchange through multiprocessing pipe

2009-04-23 Thread Jesse Noller
On Thu, Apr 23, 2009 at 5:15 AM, Michal Chruszcz wrote: > On Apr 22, 10:30 pm, Scott David Daniels > wrote: >> Michal Chruszcz wrote: >> > ... First idea, which came to my mind, was using a queue. I've got many >> > producers (all of the workers) and one consumer. Seams quite simple, >> > but it

Re: multiprocessing, pool and process crashes

2009-04-30 Thread Jesse Noller
On Wed, Apr 29, 2009 at 10:50 AM, wrote: > I want to use the multiprocessing.Pool object to run multiple tasks in > separate processes. > > The problem is that I want to call an external C function (from a > shared library, with help from ctypes) and this function tends to > crash (SIGSEGV,SIGFPE

Re: Multiprocessing Pool and functions with many arguments

2009-04-30 Thread Jesse Noller
On Wed, Apr 29, 2009 at 2:01 PM, psaff...@googlemail.com wrote: > I'm trying to get to grips with the multiprocessing module, having > only used ParallelPython before. > > based on this example: > > http://docs.python.org/library/multiprocessing.html#using-a-pool-of-workers > > what happens if I w

Re: [Python-Dev] [RELEASED] Python 3.1 beta 1

2009-05-07 Thread Jesse Noller
On Thu, May 7, 2009 at 9:12 AM, Scott David Daniels wrote: > Daniel Fetchinson wrote: >>> >>> Other features include an ordered dictionary implementation >> >> Are there plans for backporting this to python 2.x just as >> multiprocessing has been? > > Why not grab the 3.1 code and do it yourself f

Re: Get multiprocessing.Queue to do priorities

2009-05-10 Thread Jesse Noller
On Sat, May 9, 2009 at 6:11 PM, uuid wrote: > The Queue module, apparently, is thread safe, but *not* process safe. If you > try to use an ordinary Queue, it appears inaccessible to the worker process. > (Which, after all, is quite logical, since methods for moving items between > the threads of t

Re: unladen swallow: python and llvm

2009-06-04 Thread Jesse Noller
You can email these questions to the unladen-swallow mailing list. They're very open to answering questions. 2009/6/4 Luis M. González : > I am very excited by this project (as well as by pypy) and I read all > their plan, which looks quite practical and impressive. > But I must confess that I can

PyCon 2011 Registration and Financial aid open and available!

2010-12-13 Thread Jesse Noller
Just q -- http://mail.python.org/mailman/listinfo/python-list

PyCon 2011 Registration and Financial aid open and available!

2010-12-13 Thread Jesse Noller
7;s going to be huge. Financial aid is also open and available: http://us.pycon.org/2011/registration/financialaid/ Feel free to reach out to anyone on the PyCon 2011 team to ask any questions you might have. We look forward to seeing you in Atlanta. Jesse Noller PyCon 2011 -- http://mail.pyt

PyCon 2011 - Full talk and tutorial list now available, registration open!

2011-01-08 Thread Jesse Noller
As a reminder: Early Bird registration (http://us.pycon.org/2011/tickets/) closes January 17th - and we have an attendance cap of 1500 total attendees (speakers are counted against this number, and guaranteed a slot) so be sure to register today! Jesse Noller PyCon 2011 -- http://mail.python.org/mailman/listinfo/python-list

Call for Applications - PSF Sponsored Sprints

2010-07-08 Thread Jesse Noller
The PSF is happy to open our first call for applications for sprint funding! Have you ever had a group of people together to hack towards a common goal? You've hosted a sprint! Have you ever wanted to get a group of like minded Pythonistas together to hack for a day? You're going to want to hold

Re: spammers on pypi

2009-06-08 Thread Jesse Noller
On Mon, Jun 8, 2009 at 3:14 PM, Robert Kern wrote: > On 2009-06-08 07:44, Skip Montanaro wrote: >> >> On Jun 5, 1:39 pm, joep  wrote: >>> >>> Is there a way to ban spammers from pypi? >> >> Can you provide some examples?  It's possible that we can apply >> SpamBayes >> to PyPI submissions in much t

Re: Status of Python threading support (GIL removal)?

2009-06-19 Thread Jesse Noller
On Fri, Jun 19, 2009 at 12:50 PM, OdarR wrote: > On 19 juin, 16:16, Martin von Loewis > If you know that your (C) code is thread safe on its own, you can >> release the GIL around long-running algorithms, thus using as many >> CPUs as you have available, in a single process. > > what do you mean ?

Re: Status of Python threading support (GIL removal)?

2009-06-19 Thread Jesse Noller
On Fri, Jun 19, 2009 at 6:10 PM, Jure Erznožnik wrote: > On Jun 19, 11:59 pm, Jesse Noller wrote: >> On Fri, Jun 19, 2009 at 12:50 PM, OdarR wrote: >> > On 19 juin, 16:16, Martin von Loewis > >> If you know that your (C) code is thread safe on its own, you can >&g

Re: Is this a bug in multiprocessing or in my script?

2009-08-05 Thread Jesse Noller
on Linux (it should use sys.exit). Gaël Varoquaux and I > noticed this when we implemented shared memory ndarrays for numpy; we > consistently got memory leaks with System V IPC for no obvious reason. > Even after Jesse Noller was informed of the problem (about half a year > ago), the bug still

Re: Is this a bug in multiprocessing or in my script?

2009-08-05 Thread Jesse Noller
On Aug 5, 3:40 pm, sturlamolden wrote: > On 5 Aug, 21:36, sturlamolden wrote: > > >http://svn.python.org/view/python/branches/release26-maint/Lib/multip... > > >http://svn.python.org/view/python/branches/release31-maint/Lib/multip... > > http://svn.python.org/view/python/trunk/Lib/multiprocessing

Re: Is this a bug in multiprocessing or in my script?

2009-08-05 Thread Jesse Noller
On Aug 5, 4:41 pm, sturlamolden wrote: > On 5 Aug, 22:28, Jesse Noller wrote: > > >http://bugs.python.org/issue6653 > > > In the future please use the bug tracker to file and track bugs with, > > so things are not as lossy. > > Ok, sorry :) > > Also see P

Call for proposals -- PyCon 2011

2010-09-23 Thread Jesse Noller
* December 15th, 2010: Acceptance emails sent. * January 19th, 2010: Early bird registration closes. * March 9-10th, 2011: Tutorial days at PyCon. * March 11-13th, 2011: PyCon main conference. * March 14-17th, 2011: PyCon sprints days. Contact Emails: Van Lindberg (Conference

PyCon 2011 Reminder: Call for Proposals, Posters and Tutorials - us.pycon.org

2010-10-25 Thread Jesse Noller
1st, 2010: Talk proposals due. December 15th, 2010: Acceptance emails sent. January 19th, 2011: Early bird registration closes. March 9-10th, 2011: Tutorial days at PyCon. March 11-13th, 2011: PyCon main conference. March 14-17th, 2011: PyCon sprints days. Contact Emails: Van Lindberg (Conference Chair