Re: multiprocessing speedup

2015-09-29 Thread Rita
Thanks for the responses. I will create another thread to supply a more realistic example. On Tue, Sep 29, 2015 at 10:12 AM, Oscar Benjamin wrote: > On Tue, 29 Sep 2015 at 02:22 Rita wrote: > >> I am using the multiprocessing with apply_async to do some work. Each >> task t

multiprocessing speedup

2015-09-28 Thread Rita
I am using the multiprocessing with apply_async to do some work. Each task takes a few seconds but I have several thousand tasks. I was wondering if there is a more efficient method and especially when I plan to operate on a large memory arrays (numpy) Here is what I have now import multiproces

Re: general module auditing

2014-07-03 Thread Rita
On Thu, Jul 3, 2014 at 8:36 AM, Mark Lawrence wrote: > On 03/07/2014 10:27, Rita wrote: > >> >> >> >> On Thu, Jul 3, 2014 at 2:54 AM, Mark Lawrence > <mailto:breamore...@yahoo.co.uk>> wrote: >> >> On 03/07/2014 02:17, Rita wrote: >>

Re: general module auditing

2014-07-03 Thread Rita
On Thu, Jul 3, 2014 at 2:54 AM, Mark Lawrence wrote: > On 03/07/2014 02:17, Rita wrote: > >> >> On Wed, Jul 2, 2014 at 2:46 PM, Irmen de Jong > <mailto:irmen.nos...@xs4all.nl>> wrote: >> >> On 2-7-2014 4:04, Rita wrote: >> > yes,

Re: general module auditing

2014-07-02 Thread Rita
On Wed, Jul 2, 2014 at 2:46 PM, Irmen de Jong wrote: > On 2-7-2014 4:04, Rita wrote: > > yes, this helps. But I want to know who uses the module, serpent. So, > when > > I upgrade it or remove it they won't be affected adversely. > > > (Please don't top-po

Re: general module auditing

2014-07-01 Thread Rita
yes, this helps. But I want to know who uses the module, serpent. So, when I upgrade it or remove it they won't be affected adversely. On Tue, Jul 1, 2014 at 2:16 PM, Irmen de Jong wrote: > On 1-7-2014 12:38, Rita wrote: > > i work in a group of developers (15 or so) who are lo

general module auditing

2014-07-01 Thread Rita
i work in a group of developers (15 or so) who are located globally. I would like to know what modules everyone is uses if I ever have to upgrade my python. Is there mechanism which will let me see who is using what? ie, tom,matplotlib bob, pylab nancy, numpy nancy, matplotlib etc... -- ---

source code control and documentation

2014-03-02 Thread Rita
Hi, I am trying to achieve this, every time I commit to svn I automatically run hooks to test my libraries, ATM I am doing this manually cd libs PYTHONPATH=. python test_lib.py and if everything passes I do a svn commit -m 'committing code' I don't have access to my svn server so I can't crea

Re: inheriting a large python code base

2014-02-16 Thread Rita
UTC+5:30, Roy Smith wrote: > > In article Cameron Simpson wrote: > > > > On 15Feb2014 12:10, Rita wrote: > > > > i just inherited a large python code base and I would like to > optimize the > > > > code (run faster). The application is a scientific applic

inheriting a large python code base

2014-02-15 Thread Rita
hi all, i just inherited a large python code base and I would like to optimize the code (run faster). The application is a scientific application so I really don't understand the internal logic. Here is what I have done so far, profile (cProfile) and was able to see where the code was taking the

Re: mapping objects

2014-02-02 Thread Rita
Thanks for the response Cameron. No amount of 'googling' could provide me with that caliber response :-) So, it seems regardless I would need a database. On Sun, Feb 2, 2014 at 1:58 AM, Cameron Simpson wrote: > On 01Feb2014 20:46, Rita wrote: > > I want to learn m

mapping objects

2014-02-01 Thread Rita
Hello, I want to learn more about ORMs so I stumbled upon, SqlAlchemy. If i had a JSON document (or XML, CSV, etc.._) is it possible to convert it to a SQLAlchemy objects? I like the ability to query/filter ( http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html#common-filter-operators) the da

Re: data validation when creating an object

2014-01-16 Thread Rita
Thanks everyone for the replies. On Thu, Jan 16, 2014 at 1:36 AM, Cameron Simpson wrote: > On 16Jan2014 15:53, Ben Finney wrote: > > Roy Smith writes: > > > Ben Finney wrote: > > > > Who says it's frowned on to do work in the initialiser? Where are > they > > > > saying it? That seems ove

Re: data validation when creating an object

2014-01-15 Thread Rita
Unfortunately, I couldn't find the reference but I know I read it somewhere. Even with a selective search I wasn't able to find it. I think I read it in context of module/class test case writing. I will keep your responses in mind therefore I will put logic in __init__ for data validation. than

data validation when creating an object

2014-01-15 Thread Rita
I would like to do some data validation when its going to a class. class Foo(object): def __init__(self): pass I know its frowned upon to do work in the __init__() method and only declarations should be there. So, should i create a function called validateData(self) inside foo? I would ca

general ConfigParser question

2013-11-25 Thread Rita
Hi, I was wondering if the default ConfigParser can handle multi line strings (especially in the relate section) For example, if i have system.ini [Global] memory = 1024 [Process A] command = sleep arguments = 100 [Process B] command = nslookup arguments = hostA output = data [Process C] comm

Re: python process accounting

2013-04-30 Thread Rita
Exactly, its a place holder measure. Currently, I have a shell program like this. #!/usr/bin/env bash exec "$@" Any thoughts how I can put python in there? On Tue, Apr 30, 2013 at 1:54 PM, Robert Kern wrote: > On 2013-04-30 17:38, Dave Angel wrote: > >> On 04/30/201

python process accounting

2013-04-30 Thread Rita
Hi, I was wondering if it possible to write a python wrapper which will account my processes. I would like to account for all the children processes (fork) by looking at their /proc/ info. Such as memory, io, open files, stats. So, instead of me running "/bin/sleep 10", i would like to run it as

Re: Python scheduler

2013-02-21 Thread Rita
Thanks. This is what I was looking for! On Thu, Feb 21, 2013 at 9:48 AM, Adam Tauno Williams wrote: > On Wed, 2013-02-20 at 23:04 -0500, Rita wrote: > > > Here is what I am trying to do. (Currently, I am doing this in cron > > but i need much more granularity). I am trying to

Re: Python scheduler

2013-02-21 Thread Rita
8:04 PM, Rita wrote: > > Hello, > > > > Here is what I am trying to do. (Currently, I am doing this in cron but i > > need much more granularity). I am trying to run program every 20 secs and > > loop forever. I have several of these types of processes, some should run >

children process

2013-02-08 Thread Rita
hi, I currently have a bash wrapper which executes a program, something like this #!/usr/bin/env bash export LD_LIBRARY_PATH=/foo:$LD_LIBRARY_PATH exec "$@" I would like to have a python process which will do process accounting for all children, so if a process starts, I would like to get all th

building python from source

2013-01-03 Thread Rita
For those building python from source what are some tests you do to make sure the compilation and installation is up to standard. For instance here are some thing I do: Tk functionality sqlite module Python is compiled with shared object (important for wsgi) Proper preloading of python libraries (s

Re: ANN: Urwid 1.1.0 - Usability and Documentation

2012-10-28 Thread Rita
+1 great library. I suppose it will never be in the standard library? :p On Wed, Oct 24, 2012 at 5:08 AM, Jean-Michel Pichavant < jeanmic...@sequans.com> wrote: > - Original Message - > > Announcing Urwid 1.1.0 > > -- > > > > Urwid home page: > > http://excess.org

Re: unittest for system testing

2012-10-17 Thread Rita
thanks. I suppose I would need a simple example from one of these libraries. ( i typed too soon for , "no code needed" ) On Wed, Oct 17, 2012 at 8:49 PM, Mark Lawrence wrote: > On 18/10/2012 01:22, Rita wrote: > >> Hi, >> >> Currently, I use a shell script to

unittest for system testing

2012-10-17 Thread Rita
Hi, Currently, I use a shell script to test how my system behaves before I deploy an application. For instance, I check if fileA, fileB, and fileC exist and if they do I go and start up my application. This works great BUT I would like to use python and in particular unittest module to test my s

Re: best way to handle this in Python

2012-07-20 Thread Rita
OTE: preferences for comp.lang.python are to follow the RFC on > "netiquette" -- that is, post comments /under/ quoted material, trimming > what is not relevant... I've restructured this reply to match} > > On Thu, 19 Jul 2012 21:28:12 -0400, Rita > declaimed the follo

Re: best way to handle this in Python

2012-07-19 Thread Rita
Using linux 2.6.31; Python 2.7.3. I am not necessary looking for code just a pythonic way of doing it. Eventually, I would like to graph the data using matplotlib On Thu, Jul 19, 2012 at 8:52 PM, Dave Angel wrote: > On 07/19/2012 07:51 PM, Rita wrote: > > Hello, > > > >

best way to handle this in Python

2012-07-19 Thread Rita
Hello, I have data in many files (/data/year/month/day/) which are named like YearMonthDayHourMinute.gz. I would like to build a data structure which can easily handle querying the data. So for example, if I want to query data from 3 weeks ago till today, i can do it rather quickly. each YearMon

recompiling standard modules

2012-07-09 Thread Rita
I noticed that active state python Tk inter isnt compiled with --enable-threads therefore I would like to recompile the module with a new version of TCL/TK which is compiled with threads. How can I do this? -- --- Get your facts first, then you can distort them as you please.-- -- http://mail.

other languages API to python

2012-05-24 Thread Rita
Hello, A vendor provided a C, C++ and Java API for a application. They dont support python so I would like to create a library for it. My question is, how hard/easy would it be to create something like this? Is there a simple HOWTO or examples I can follow? Can someone shed home light on this? TI

standard module xpath

2012-05-18 Thread Rita
Hello, Does python ship with xpath query language support? Certainly, I can use lxml but I prefer something in the standard modules list. -- --- Get your facts first, then you can distort them as you please.-- -- http://mail.python.org/mailman/listinfo/python-list

bash/shell to python

2012-05-16 Thread Rita
Hello, I currently build a lot of interfaces/wrappers to other applications using bash/shell. One short coming for it is it lacks a good method to handle arguments so I switched to python a while ago to use 'argparse' module. Its a great complement to subprocess module. I was wondering if there is

compiling Tkinter

2012-05-15 Thread Rita
Hello, I understand Tkinter is part of the python distribution but for me it always fails when I try to load the module. I get: >>> import Tkinter ... import _tkinter # if this fails your Python may not be configured for Tk ImportError: No module named _tkinter So, here is how I am compiling tc

python library to generate restructured text

2012-03-03 Thread Rita
Hello, I am using sqlite3 modules to get data out of a table. I would like to dump the table into a restructured-text (docutil) format. I was wondering if there was a library which would accomplish this. -- --- Get your facts first, then you can distort them as you please.-- -- http://mail.pyt

Re: standalone python web server

2012-02-09 Thread Rita
yes, I would like to use a framework. I like the twisted method the user posted. Are there any examples of it using a framework, get/post, etc..? On Thu, Feb 9, 2012 at 6:28 AM, Thomas Bach wrote: > Rita writes: > > > I am building a small intranet website and I would like to us

standalone python web server

2012-02-08 Thread Rita
I am building a small intranet website and I would like to use Python. I was wondering if there was a easy and medium performance python based web server available. I would like to run it on port :8080 since I wont have root access also I prefer something easy to deploy meaning I would like to move

working with a large file

2011-09-12 Thread Rita
I have a large file, 18gb uncompressed, and I would like to know what is the preferred method to read this file for random access. I have several processes reading the file which different calculate arguments. My server has 64gb of memory. Not sure what is the preferred way to do this? -- ---

code generation

2011-08-02 Thread Rita
Hello, This isn't much of a python question but a general algorithm question. I plan to input the following string and I would like to generate something like this. input: a->(b,c)->d output: parent a, child b c parent b c child d Are there any libraries or tools which will help me evaluate ite

complex data structures in python

2011-05-02 Thread Rita
Hello, I was wondering if anyone has any documentation/recipes for implementing complex data structures. For instance, if you had a dictionary with a list inside a list inside a set. -- --- Get your facts first, then you can distort them as you please.-- -- http://mail.python.org/mailman/listin

PyCon

2011-03-11 Thread Rita
Hello, I noticed the old presentations at PyCon aren't avaliable anymore. http://us.pycon.org/2010/ http://us.pycon.org/2009/ Does anyone know where else I can get these presentations? -- --- Get your facts first, then you can distort them as you please.-- -- http://mail.python.org/mailman/lis

Re: parsing a file for analysis

2011-02-26 Thread Rita
, Feb 26, 2011 at 10:53 AM, Martin Gregorie wrote: > On Sat, 26 Feb 2011 16:29:54 +0100, Andrea Crotti wrote: > > > Il giorno 26/feb/2011, alle ore 06.45, Rita ha scritto: > > > >> I have a large text (4GB) which I am parsing. > >> > >> I am readi

Re: parsing a file for analysis

2011-02-26 Thread Rita
. And next time I read thru this file, I can skip all of Jan 01, 2011 On Sat, Feb 26, 2011 at 10:29 AM, Andrea Crotti wrote: > > Il giorno 26/feb/2011, alle ore 06.45, Rita ha scritto: > > > I have a large text (4GB) which I am parsing. > > > > I am reading the file

parsing a file for analysis

2011-02-25 Thread Rita
I have a large text (4GB) which I am parsing. I am reading the file to collect stats on certain items. My approach has been simple, for row in open(file): if "INFO" in row: line=row.split() user=line[0] host=line[1] __time=line[2] ... I was wondering if there is a framewor

Re: subprocess pipe question

2011-02-25 Thread Rita
The results look right! I did a rather large test and the checksum passed. I will hold off any speed ups as you suggested. On Wed, Feb 23, 2011 at 8:37 PM, Rob Williscroft wrote: > Rita wrote in > news:AANLkTi=88dcpm_kqrs2g620obsnxz0majubfwpeme...@mail.gmail.com in > gmane.comp.pytho

Re: subprocess pipe question

2011-02-23 Thread Rita
This is a good solution thanks. You should wiki this somewhere. For extra points is there a way to speed up the p.stdout.read(bufsize) ? On Tue, Feb 22, 2011 at 7:57 PM, Rob Williscroft wrote: > Rita wrote in > news:AANLkTi=w95gxosc1tkt2bntgjqys1cbmdnojhokq4...@mail.gmail.

Re: subprocess pipe question

2011-02-22 Thread Rita
at 3:44 PM, Rita wrote: > >> I have a process like this, >> >> def run(cmd): >> #cmd=a process which writes a lot of data. Binary/ASCII data >> p=subprocess.Popen(cmd,stdout=subprocess.PIPE) >> >> I would like to get cmd's return code so I am doing

subprocess pipe question

2011-02-22 Thread Rita
I have a process like this, def run(cmd): #cmd=a process which writes a lot of data. Binary/ASCII data p=subprocess.Popen(cmd,stdout=subprocess.PIPE) I would like to get cmd's return code so I am doing this, def run(cmd): p=subprocess.Popen(cmd,stdout=subprocess.PIPE) rc=p.poll() print

Converting from shell to python

2011-02-21 Thread Rita
Hello, I have been using shell for a "long" time and I decided to learn python recently. So far I am liking it a lot especially the argparse module which makes my programs more professional like. Currently, I am rewriting my bash scripts to python so I came across a subprocess and environment pr