Re: organizing many python scripts, in a large corporate environment.

[eryksun ()]

On Tuesday, March 15, 2011 12:44:48 AM UTC-4, bukzor wrote:
>
> Currently it requires either: 1) no symlinks to scripts or 2)
> installation of the pathtools to site-packages.

An executable with a unique name on the system PATH could be executed it as a 
subprocess that pipes the configured base directory back via stdout. Then you 
can just do site.addsitedir(base).
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: possible to run a python script without installing python?

[anpeo]

On Mar 16, 1:15 am, Terry Reedy  wrote:
> On 3/15/2011 4:58 PM, davidj411 wrote:
>
> > it seems that if I copy the python.exe binary and the folders
> > associated with it to a server without python, i can run python.
> > does anyone know which files are important to copy and which can be
> > omitted?
>
> For the 3.2 Windows installation, you should be able to omit Doc,
> Lib/test, Lib/turtledemo, and Tools. Three of those are optional
> installs anyway. Lib/idlelib could almost certainly go if not running
> IDLE from the server (which would be unlikely). If not using tcl/tk,
> several files in DLLs, Lib, and libs and the tcl directory can go.
>
> Translate as needed for *nix.
>
> --
> Terry Jan Reedy

I use cx_freeze witch also works with Python 3.x
Test it, for me it was easier than to figure out what to copy

http://cx-freeze.sourceforge.net/

// Anders
-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] OpenOpt Suite release 0.33

[dmitrey]

Hi all, I'm glad to inform you about new release 0.33 of our
completely free (license: BSD) cross-platform software:

OpenOpt:

* cplex has been connected
* New global solver interalg with guarantied precision, competitor
to LGO, BARON, MATLAB's intsolver and Direct (also can work in inexact
mode)
* New solver amsg2p for unconstrained medium-scaled NLP and NSP

FuncDesigner:

* Essential speedup for automatic differentiation when vector-
variables are involved, for both dense and sparse cases
* Solving MINLP became available
* Add uncertainty analysis
* Add interval analysis
* Now you can solve systems of equations with automatic
determination is the system linear or nonlinear (subjected to given
set of free or fixed variables)
* FD Funcs min and max can work on lists of oofuns
* Bugfix for sparse SLE (system of linear equations), that slowed
down computation time and demanded more memory
* New oofuns angle, cross
* Using OpenOpt result(oovars) is available, also, start points
with oovars() now can be assigned easier

SpaceFuncs (2D, 3D, N-dimensional geometric package with abilities for
parametrized calculations, solving systems of geometric equations and
numerical optimization with automatic differentiation):

* Some bugfixes

DerApproximator:

* Adjusted with some changes in FuncDesigner

For more details visit our site http://openopt.org.

Regards, Dmitrey.
-- 
http://mail.python.org/mailman/listinfo/python-list

Problem with command-line registeration with PyPI in Windows 7

[melhosseiny]

When I type c:\python32\python.exe setup.py register and press enter I
get the following:

running register
running check
We need to know who you are, so please choose either:
 1. use your existing login,
 2. register as a new user,
 3. have the server generate a new password for you (and email it to
you), or
 4. quit
Your selection [default 1]:

So i type 1 and press enter and i get:

Please choose one of the four options!

Why is this happening? It doesn't care what i type. It's always the
same result.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multiprocessing.Process Daemonic Behavior

[John L. Stephens]

On 3/15/2011 11:19 PM, James Mills wrote:

On Wed, Mar 16, 2011 at 12:34 PM, John L. Stephens
  wrote:

I would have expected the daemonic children processes to terminate with the
parent process, regardless of how the parent process terminates, either
normally or forcefully.

As I understand it. If you forcibly kill the parent process
with the KILL signal then any child procesases that were
created become zombies. You also can't handle the KILL
signal in your application (nor can the multiprocessing library)
and so it therefore cannot cleanup  and terminate any child
processes in the normal way.

cheers
James

So as I have contemplated this in the wee hours of the morning (risking 
zombie status myself I might add),  my rationale for the behavior I am 
seeing is thus:


As the parent process terminates 'normally' (either through normal 
termination or SIGINT termination), mulitprocessing steps in and 
performs child process cleanup via the x.terminate() method.  If the 
parent terminates any other way, multiprocessing doesn't have the 
opportunity to cleanup.


I would have thought (silly me) the child process would also monitor his 
parent process, and when the child can no longer 'see' or 'sense the 
presence' of its parent, it too would terminate.


I can certainly make the children monitor the parent as well, I was just 
hoping I wouldn't have to put in additional code and overhead.


John



--
http://mail.python.org/mailman/listinfo/python-list

Re: alphanumeric list

[yqyq22]

On Mar 15, 2:15 pm, yqyq22  wrote:
> On Mar 15, 11:02 am, Laurent Claessens  wrote:
>
>
>
>
>
> > Le 15/03/2011 09:10, yqyq22 a crit :
>
> > > Hi all,
> > > I would like to put an alphanumeric string like this one
> > > EE472A86441AF2E629DE360 in a list, then iterate inside the entire
> > > string lenght and change each digit with a random digit.
> > > Do u have some suggestion? thanks a lot
>
> > This can be a way to begin :
>
> > s="EE472A86441AF2E629DE360"
> > for a in s:
> >     print a
>
> > The following is a very basic trick to build a list  containing the
> > elements of the string 
>
> > s="EE472A86441AF2E629DE360"
> > t=[]
> > for a in s:
> >     t.append(a)
>
> > All that you have to change is t.append(a) into something that
> > tests if a is a number and append something else in that case.
>
> > Depending what you want, you can even do
>
> > s.replace("1","ONE").replace("2","TWO").replace("3","FIVE")
>
> > This is however far from being random ;)
>
> > Have good tests
> > Laurent
>
> Laurent thanks a lot very much!!! really appreciated it.. now i have a
> good start point thanks again- Hide quoted text -
>
> - Show quoted text -

Something like that but it append A to each element of the string

string = "EE472B"
t = []
for x in string[:]:
print t.append("A")   #i don't want to append A, the final
result is EAEA4A7A2ABA
# i would like to substitute each element with a random
string

How i can use the random module in this case? Thanks

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with command-line registeration with PyPI in Windows 7

[Steven D'Aprano]

On Wed, 16 Mar 2011 01:55:31 -0700, melhosseiny wrote:

> When I type c:\python32\python.exe setup.py register and press enter I
> get the following:
[...]
> Why is this happening? It doesn't care what i type. It's always the same
> result.

I found a similar (possibly the same) problem. I eventually worked around 
it by creating a config file with my username and password:


[distutils]
index-servers =
pypi

[pypi]
username:
password:


How you would do the same thing under Windows, I don't know.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: alphanumeric list

[Laurent Claessens]

string = "EE472B"
t = []
for x in string[:]:
 print t.append("A")   #i don't want to append A, the final
result is EAEA4A7A2ABA


As it, it prints "None" 6 times. Then t is ['A','A','A','A','A']

What you have to do is to read about basic python. Search for python 
tutorial on the net.


Then try the following exercices :

Exercises

1
print the first 1000 intergers

2
print the sine of 1 ... 1000

3
print the intergers x between 1 and 1000 such that cos(x) is between 0 
and 0.5



Have a good afternoon
Laurent
--
http://mail.python.org/mailman/listinfo/python-list

Re: alphanumeric list

[Laurent Claessens]

string = "EE472B"
t = []
for x in string[:]:
 print t.append("A")   #i don't want to append A, the final
result is EAEA4A7A2ABA


As it, it prints "None" 6 times. Then t is ['A','A','A','A','A']

What you have to do is to read about basic python. Search for python 
tutorial on the net.


Then try the following exercices :

Exercises

1
print the first 1000 intergers

2
print the sine of 1 ... 1000

3
print the intergers x between 1 and 1000 such that cos(x) is between 0 
and 0.5



Have a good afternoon
Laurent
--
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with command-line registeration with PyPI in Windows 7

[melhosseiny]

On Mar 16, 2:51 pm, Steven D'Aprano  wrote:
> On Wed, 16 Mar 2011 01:55:31 -0700, melhosseiny wrote:
> > When I type c:\python32\python.exe setup.py register and press enter I
> > get the following:
> [...]
> > Why is this happening? It doesn't care what i type. It's always the same
> > result.
>
> I found a similar (possibly the same) problem. I eventually worked around
> it by creating a config file with my username and password:
>
> [distutils]
> index-servers =
>     pypi
>
> [pypi]
> username:
> password:
>
> How you would do the same thing under Windows, I don't know.
>
> --
> Steven

Thank you. It worked for me too. I added .pypirc file with my username
and password in my home folder (i.e. C:\Users\*).
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: organizing many python scripts, in a large corporate environment.

[booklover]

> I'm going to try to get our solution open-sourced, then I'll get your
> feedback on it.

Thanks bukzor! I think that it would be very helpful to have a library
like this available.

In the longer term, what do people think about the possibility of
writing up a PEP to fix this problem in the core language? Anyone have
any ideas on the cleanest way to achieve this goal?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: possible to run a python script without installing python?

[David Jackson]

i need to run a python script on any arbitrary server and don't want to do
an actual installation. i figured i could do a clean install on my machine
and install whatever libraries would be needed, then zip them all up for
remote deployment. to avoid bloating, i am wondering which files i can
safely omit.


On Tue, Mar 15, 2011 at 8:48 PM, Katie T  wrote:

>
> On Tue, Mar 15, 2011 at 8:58 PM, davidj411  wrote:
>
>> it seems that if I copy the python.exe binary and the folders
>> associated with it to a server without python, i can run python.
>> does anyone know which files are important to copy and which can be
>> omitted?
>>
>> i know about py2exe and have had no luck with it.
>
>
> What's the reason for wanting to avoid installing Python? - are you just
> trying to save disk space?
>
> If it's a case of not having admin rights, you can just copy the Python
> directory, I don't believe it has any dependencies anywhere else.
>
> Katie
> --
> CoderStack
> http://www.coderstack.co.uk
> The Software Developer Job Board
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Memory Usage of Strings

[Amit Dev]

I'm observing a strange memory usage pattern with strings. Consider
the following session. Idea is to create a list which holds some
strings so that cumulative characters in the list is 100MB.

>>> l = []
>>> for i in xrange(10):
...  l.append(str(i) * (1000/len(str(i

This uses around 100MB of memory as expected and 'del l' will clear that.


>>> for i in xrange(2):
...  l.append(str(i) * (5000/len(str(i

This is using 165MB of memory. I really don't understand where the
additional memory usage is coming from.

If I reduce the string size, it remains high till it reaches around
1000. In that case it is back to 100MB usage.

Python 2.6.4 on FreeBSD.

Regards,
Amit
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: function annotations in open source projects

[Filip Gruszczyński]

Thanks for the links.

-- 
Filip Gruszczyński
-- 
http://mail.python.org/mailman/listinfo/python-list

strange QLineEdit problem in PyQt

[taco]

hi,
I have a form containing a few QLineedits and I wish to retrieve the 
contents only after pressing a QPushButton.
a looks like:
class Contract(QtGui.QWidget):
  def __init__(self,fileInfo, parent=None):
QtGui.QWidget.__init__(self,parent)
self.name = QtGui.QLineEdit("")
.
self.createDoc = QtGui.QPushButton("do it")
self.connect(createDoc,QtCore.SIGNAL("clicked()"),self.Foo)
mainLayout.addWidget(self.name)
mainLayout.addWidget(sef.createDoc)

 def Foo(self):
   print self.name.text()

when executing this I get only a few characters back from the line. It's 
incomplete. How to retrieve the complete string? I don't need a signal for 
every typed character.

taco

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: strange QLineEdit problem in PyQt

[taco]

taco wrote:

ah, I solved it myself. for completeness reasons. I had upgraded python for 
reportlab which needed this python version, while still using a pyqt which 
was compiled for 2.6. After compilation and changing pythonpath to the right 
site-packages dir it's working


> hi,
> I have a form containing a few QLineedits and I wish to retrieve the
> contents only after pressing a QPushButton.
> a looks like:
> class Contract(QtGui.QWidget):
>   def __init__(self,fileInfo, parent=None):
> QtGui.QWidget.__init__(self,parent)
> self.name = QtGui.QLineEdit("")
> .
> self.createDoc = QtGui.QPushButton("do it")
> self.connect(createDoc,QtCore.SIGNAL("clicked()"),self.Foo)
> mainLayout.addWidget(self.name)
> mainLayout.addWidget(sef.createDoc)
> 
>  def Foo(self):
>print self.name.text()
> 
> when executing this I get only a few characters back from the line. It's
> incomplete. How to retrieve the complete string? I don't need a signal for
> every typed character.
> 
> taco

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: strange QLineEdit problem in PyQt

[Rafael Durán Castañeda]

Just one remark about this line:

QtGui.QWidget.__init__(self, parent)

I think you should use this:

super(Contract, self).__init__(parent)

so you will be following PEP 3135


2011/3/16 taco 

> taco wrote:
>
> ah, I solved it myself. for completeness reasons. I had upgraded python for
> reportlab which needed this python version, while still using a pyqt which
> was compiled for 2.6. After compilation and changing pythonpath to the
> right
> site-packages dir it's working
>
>
> > hi,
> > I have a form containing a few QLineedits and I wish to retrieve the
> > contents only after pressing a QPushButton.
> > a looks like:
> > class Contract(QtGui.QWidget):
> >   def __init__(self,fileInfo, parent=None):
> > QtGui.QWidget.__init__(self,parent)
> > self.name = QtGui.QLineEdit("")
> > .
> > self.createDoc = QtGui.QPushButton("do it")
> > self.connect(createDoc,QtCore.SIGNAL("clicked()"),self.Foo)
> > mainLayout.addWidget(self.name)
> > mainLayout.addWidget(sef.createDoc)
> >
> >  def Foo(self):
> >print self.name.text()
> >
> > when executing this I get only a few characters back from the line. It's
> > incomplete. How to retrieve the complete string? I don't need a signal
> for
> > every typed character.
> >
> > taco
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

PostgreSQL vs MySQL (was Re: How to handle sockets - easily?)

[Aahz]

In article ,
William Ahern   wrote:
>
>I think that there's an asynchronous all-Python MySQL library, but I'm not
>sure. Maybe one day I can open source my asynchronous MySQL C library. (I
>always recommend people to use PostgreSQL, though; which is superior in
>almost every way, especially the C client library and the wire protocol.)

Can you point at a reference for the latter?  I have been trying to
convince my company that PG is better than MySQL.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"Beware of companies that claim to be like a family.  They might not be
lying."  --Jill Lundquist
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Memory Usage of Strings

[John Gordon]

In  Amit Dev 
 writes:

> I'm observing a strange memory usage pattern with strings. Consider
> the following session. Idea is to create a list which holds some
> strings so that cumulative characters in the list is 100MB.

> >>> l = []
> >>> for i in xrange(10):
> ...  l.append(str(i) * (1000/len(str(i

> This uses around 100MB of memory as expected and 'del l' will clear that.

> >>> for i in xrange(2):
> ...  l.append(str(i) * (5000/len(str(i

> This is using 165MB of memory. I really don't understand where the
> additional memory usage is coming from.

> If I reduce the string size, it remains high till it reaches around
> 1000. In that case it is back to 100MB usage.

I don't know anything about the internals of python storage -- overhead,
possible merging of like strings, etc.  but some simple character counting
shows that these two loops do not produce the same number of characters.

The first loop produces:

Ten single-digit values of i which are repeated 1000 times for a total of
1 characters;

Ninety two-digit values of i which are repeated 500 times for a total of
45000 characters;

Nine hundred three-digit values of i which are repeated 333 times for a
total of 299700 characters;

Nine thousand four-digit values of i which are repeated 250 times for a
total of 225 characters;

Ninety thousand five-digit values of i which are repeated 200 times for
a total of 1800 characters.

All that adds up to a grand total of 20604700 characters.

Or, to condense the above long-winded text in table form:

range num digits 1000/len(str(i))  total chars
0-910 1  10001
10-99  90 2   50045000
100-999   900 3   333   299700
1000-9000 4   250  225
1-9 9 5   200 1800
  
  grand total chars   20604700

The second loop yields this table:

range num digits 5000/len(str(i))  total bytes
0-910 1  50005
10-99  90 2  2500   225000
100-999   900 3  1666  1499400
1000-9000 4  1250 1125
1-1 1 5  1000 1000
  
  grand total chars   23024400

The two loops do not produce the same numbers of characters, so I'm not
surprised they do not consume the same amount of storage.

P.S.: Please forgive me if I've made some basic math error somewhere.

-- 
John Gordon   A is for Amy, who fell down the stairs
gor...@panix.com  B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: possible to run a python script without installing python?

[Santoso Wijaya]

py2exe does this for you...

~/santa


On Wed, Mar 16, 2011 at 8:02 AM, David Jackson  wrote:

> i need to run a python script on any arbitrary server and don't want to do
> an actual installation. i figured i could do a clean install on my machine
> and install whatever libraries would be needed, then zip them all up for
> remote deployment. to avoid bloating, i am wondering which files i can
> safely omit.
>
>
> On Tue, Mar 15, 2011 at 8:48 PM, Katie T  wrote:
>
>>
>> On Tue, Mar 15, 2011 at 8:58 PM, davidj411  wrote:
>>
>>> it seems that if I copy the python.exe binary and the folders
>>> associated with it to a server without python, i can run python.
>>> does anyone know which files are important to copy and which can be
>>> omitted?
>>>
>>> i know about py2exe and have had no luck with it.
>>
>>
>> What's the reason for wanting to avoid installing Python? - are you just
>> trying to save disk space?
>>
>> If it's a case of not having admin rights, you can just copy the Python
>> directory, I don't believe it has any dependencies anywhere else.
>>
>> Katie
>> --
>> CoderStack
>> http://www.coderstack.co.uk
>> The Software Developer Job Board
>>
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Memory Usage of Strings

[Amit Dev]

sum(map(len, l)) =>  8200 for 1st case and 9100 for 2nd case.
Roughly 100MB as I mentioned.

On Wed, Mar 16, 2011 at 11:21 PM, John Gordon  wrote:
> In  Amit Dev 
>  writes:
>
>> I'm observing a strange memory usage pattern with strings. Consider
>> the following session. Idea is to create a list which holds some
>> strings so that cumulative characters in the list is 100MB.
>
>> >>> l = []
>> >>> for i in xrange(10):
>> ...  l.append(str(i) * (1000/len(str(i
>
>> This uses around 100MB of memory as expected and 'del l' will clear that.
>
>> >>> for i in xrange(2):
>> ...  l.append(str(i) * (5000/len(str(i
>
>> This is using 165MB of memory. I really don't understand where the
>> additional memory usage is coming from.
>
>> If I reduce the string size, it remains high till it reaches around
>> 1000. In that case it is back to 100MB usage.
>
> I don't know anything about the internals of python storage -- overhead,
> possible merging of like strings, etc.  but some simple character counting
> shows that these two loops do not produce the same number of characters.
>
> The first loop produces:
>
> Ten single-digit values of i which are repeated 1000 times for a total of
> 1 characters;
>
> Ninety two-digit values of i which are repeated 500 times for a total of
> 45000 characters;
>
> Nine hundred three-digit values of i which are repeated 333 times for a
> total of 299700 characters;
>
> Nine thousand four-digit values of i which are repeated 250 times for a
> total of 225 characters;
>
> Ninety thousand five-digit values of i which are repeated 200 times for
> a total of 1800 characters.
>
> All that adds up to a grand total of 20604700 characters.
>
> Or, to condense the above long-winded text in table form:
>
> range         num digits 1000/len(str(i))  total chars
> 0-9            10 1      1000                    1
> 10-99          90 2       500                    45000
> 100-999       900 3       333                   299700
> 1000-    9000 4       250                  225
> 1-9 9 5       200                 1800
>                                              
>                          grand total chars   20604700
>
> The second loop yields this table:
>
> range         num digits 5000/len(str(i))  total bytes
> 0-9            10 1      5000                    5
> 10-99          90 2      2500                   225000
> 100-999       900 3      1666                  1499400
> 1000-    9000 4      1250                 1125
> 1-1 1 5      1000                 1000
>                                              
>                          grand total chars   23024400
>
> The two loops do not produce the same numbers of characters, so I'm not
> surprised they do not consume the same amount of storage.
>
> P.S.: Please forgive me if I've made some basic math error somewhere.
>
> --
> John Gordon                   A is for Amy, who fell down the stairs
> gor...@panix.com              B is for Basil, assaulted by bears
>                                -- Edward Gorey, "The Gashlycrumb Tinies"
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

pip install, permission problem

[Jabba Laci]

Hi,

I'm trying to install a pypi package with pip. The installation is
done, but the permissions are not set correctly, thus I cannot import
it as a normal user.
Example:
sudo pip install termcolor
/usr/local/lib/python2.6/dist-packages/termcolor.py is created with
permissions 600

I can post-correct it manually, but there is something wrong going on
here... I have a hint but I'm not sure: as a normal user, I have
"umask 077" in my .bashrc. When I do sudo, maybe this setting is
inherited? But even if it's the case, pip install should change the
permissions correctly IMO.

Laszlo
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Print to an IPP printer (pkipplib?)

[Adam Tauno Williams]

On Fri, 2010-10-15 at 15:28 -0400, Adam Tauno Williams wrote:
> I've found the module pkipplib which seems to work well for things like
> interrogating an IPP (CUPS) server.  But is there a way to send a print
> job to an IPP print queue? [and no, the local system knows nothing about
> the print architecture so popenlp is not an option].  I just want to
> send the data from a file handle to a remote IPP queue as a print job.

I've never achieved printing to an IPP queue;  but printing to an
LPR/LPD queue (which CUPS also supports) turned out to be pretty easy.



Printing without any subprocess thunkery.


-- 
http://mail.python.org/mailman/listinfo/python-list

ECCOMAS VipIMAGE 2011 – SUBMISSION DEADLINE EXTENDED

[tava...@fe.up.pt]

---

International ECCOMAS Thematic Conference VipIMAGE 2011 - III ECCOMAS
THEMATIC
CONFERENCE ON COMPUTATIONAL VISION AND MEDICAL IMAGE PROCESSING
12-14th October 2011, Olhão, Algarve, Portugal
www.fe.up.pt/~vipimage

SUBMISSION DEADLINE EXTENDED to March 31

We would appreciate if you could distribute this information by your
colleagues and co-workers.

---

Dear Colleague,

We are pleased to announce an extension of the deadline for abstract
submission to the International Conference VipIMAGE 2011 - III ECCOMAS
THEMATIC CONFERENCE ON COMPUTATIONAL VISION AND MEDICAL IMAGE
PROCESSING (www.fe.up.pt/~vipimage), to March 31.

Possible Topics (not limited to)

- Signal and Image Processing
- Computational Vision
- Medical Imaging
- Physics of Medical Imaging
- Tracking and Analysis of Movement
- Simulation and Modeling
- Image Acquisition
- Shape Reconstruction
- Objects Segmentation, Matching, Simulation
- Data Interpolation, Registration, Acquisition and Compression
- 3D Vision
- Virtual Reality
- Software Development for Image Processing and Analysis
- Computer Aided Diagnosis, Surgery, Therapy, and Treatment
- Computational Bioimaging and Visualization
- Telemedicine Systems and their Applications


Invited Lecturers

- Armando J. Pinho - University of Aveiro, Portugal
- Irene M. Gamba - The University of Texas at Austin, USA
- Marc Pollefeys - ETH Zurich, Switzerland
- Marc Thiriet - Universite Pierre et Marie Curie (Paris VI), France
- Xavier Roca Marvà - Autonomous University of Barcelona, Spain
- Stan Sclaroff - Boston University, USA


Thematic Sessions

Proposals to organize Thematic Session within VipIMAGE 2011 are
welcome and should be submitted by email to the conference co-chairs
(tava...@fe.up.pt, rna...@fe.up.pt)


Thematic Sessions Confirmed

- Simultaneous MR-PET imaging
- Satellite image analysis for environmental risk assessment
- Dental Imaging and Processing Techniques
- Digital Mammography
- Imaging of Biological Flows: trends and challenges
- Imaging techniques applied to soft tissue Biomechanics


Publications

The proceedings book will be published by the Taylor & Francis Group
and indexed by Thomson Reuters Conference Proceedings Citation Index,
IET Inspect and Elsevier Scopus.
A book with 20 invited works from the best ones presented in
VipIMAGE2011 (extended versions) will be published by Springer.
The organizers will encourage the submission of extended versions of
the accepted papers to related International Journals; in particular,
for special issues dedicated to the conference.


Important dates

- Deadline for Extended Abstracts: 31st March 2011 (postponed)
- Authors Notification: 15th April 2011
- Deadline for Full Papers: 15th June 2011


Awards

"best paper award" and "best student paper award" are going to be
given to the author(s) of two papers presented at the conference,
selected by the Organizing Committee based on the best combined marks
from the Scientific Committee and Session Chairs.


We are looking forward to see you in Algarve next October.

Kind regards,

João Manuel R. S. Tavares
Renato Natal Jorge
(conference co-chairs)

PS. For further details please see the conference website at:
www.fe.up.pt/~vipimage
-- 
http://mail.python.org/mailman/listinfo/python-list

os.path.walk() to get full path of all files

[dude]

My goal is create a list of absolute paths for all files in a given
directory (any number of levels deep).

root
dir1
file1
file2
dir2
file3
dir3
-dir4
--file4
file5

So the above would return:
[root/dir1/file1, root/dir1/file2, root/dir1/dir2/file3, etc...]

I've been trying different ways of using os.path.walk() for that, but
I can't find an elegant way.  Anyone know of something simple to
accomplish that?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Memory Usage of Strings

[Santoso Wijaya]

??

Python 2.7.1 (r271:86832, Nov 27 2010, 17:19:03) [MSC v.1500 64 bit (AMD64)]
on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> L = []
>>> for i in xrange(10):
... L.append(str(i) * (1000 / len(str(i
...
>>> sys.getsizeof(L)
824464
>>> L = []
>>> for i in xrange(2):
... L.append(str(i) * (5000 / len(str(i
...
>>> sys.getsizeof(L)
178024
>>>

~/santa


On Wed, Mar 16, 2011 at 11:20 AM, Amit Dev  wrote:

> sum(map(len, l)) =>  8200 for 1st case and 9100 for 2nd case.
> Roughly 100MB as I mentioned.
>
> On Wed, Mar 16, 2011 at 11:21 PM, John Gordon  wrote:
> > In  Amit Dev <
> amit...@gmail.com> writes:
> >
> >> I'm observing a strange memory usage pattern with strings. Consider
> >> the following session. Idea is to create a list which holds some
> >> strings so that cumulative characters in the list is 100MB.
> >
> >> >>> l = []
> >> >>> for i in xrange(10):
> >> ...  l.append(str(i) * (1000/len(str(i
> >
> >> This uses around 100MB of memory as expected and 'del l' will clear
> that.
> >
> >> >>> for i in xrange(2):
> >> ...  l.append(str(i) * (5000/len(str(i
> >
> >> This is using 165MB of memory. I really don't understand where the
> >> additional memory usage is coming from.
> >
> >> If I reduce the string size, it remains high till it reaches around
> >> 1000. In that case it is back to 100MB usage.
> >
> > I don't know anything about the internals of python storage -- overhead,
> > possible merging of like strings, etc.  but some simple character
> counting
> > shows that these two loops do not produce the same number of characters.
> >
> > The first loop produces:
> >
> > Ten single-digit values of i which are repeated 1000 times for a total of
> > 1 characters;
> >
> > Ninety two-digit values of i which are repeated 500 times for a total of
> > 45000 characters;
> >
> > Nine hundred three-digit values of i which are repeated 333 times for a
> > total of 299700 characters;
> >
> > Nine thousand four-digit values of i which are repeated 250 times for a
> > total of 225 characters;
> >
> > Ninety thousand five-digit values of i which are repeated 200 times for
> > a total of 1800 characters.
> >
> > All that adds up to a grand total of 20604700 characters.
> >
> > Or, to condense the above long-winded text in table form:
> >
> > range num digits 1000/len(str(i))  total chars
> > 0-910 1  10001
> > 10-99  90 2   50045000
> > 100-999   900 3   333   299700
> > 1000-9000 4   250  225
> > 1-9 9 5   200 1800
> >  
> >  grand total chars   20604700
> >
> > The second loop yields this table:
> >
> > range num digits 5000/len(str(i))  total bytes
> > 0-910 1  50005
> > 10-99  90 2  2500   225000
> > 100-999   900 3  1666  1499400
> > 1000-9000 4  1250 1125
> > 1-1 1 5  1000 1000
> >  
> >  grand total chars   23024400
> >
> > The two loops do not produce the same numbers of characters, so I'm not
> > surprised they do not consume the same amount of storage.
> >
> > P.S.: Please forgive me if I've made some basic math error somewhere.
> >
> > --
> > John Gordon   A is for Amy, who fell down the stairs
> > gor...@panix.com  B is for Basil, assaulted by bears
> >-- Edward Gorey, "The Gashlycrumb Tinies"
> >
> > --
> > http://mail.python.org/mailman/listinfo/python-list
> >
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.path.walk() to get full path of all files

[Alexander Kapps]

On 16.03.2011 20:41, dude wrote:

My goal is create a list of absolute paths for all files in a given
directory (any number of levels deep).

root
dir1
file1
file2
dir2
file3
dir3
-dir4
--file4
file5

So the above would return:
[root/dir1/file1, root/dir1/file2, root/dir1/dir2/file3, etc...]

I've been trying different ways of using os.path.walk() for that, but
I can't find an elegant way.  Anyone know of something simple to
accomplish that?


Try this:

file_list = []
for root, _, filenames in os.walk(root_path):
for filename in filenames:
file_list.append(os.path.join(root, filename))

for x in file_list:
print x
--
http://mail.python.org/mailman/listinfo/python-list

Re: OT: processes, terminals and file descriptors on *nix

[Alexander Kapps]

On 13.03.2011 01:50, Nobody wrote:


I don't have any links. If you want to understand the core Unix API, the
best reference I know of is Stevens ("Advanced Programming in the Unix
Environment", by W. Richard Stevens). Unfortunately, it's not cheap.

In spite of the title, it doesn't assume any prior Unix knowledge; the
"Advanced" mainly refers to the fact that it's the opposite of the "Learn
X in Five Minutes" books, i.e. it's thorough. That's how it comes to>700
pages while only covering the core API (files, processes, and terminals;
no sockets, no X, no I18N, ...).



Thanks, Nobody and Dan. I'll have a look at the book.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Memory Usage of Strings

[eryksun ()]

On Wednesday, March 16, 2011 2:20:34 PM UTC-4, Amit Dev wrote:
>
> sum(map(len, l)) =>  8200 for 1st case and 9100 for 2nd case.
> Roughly 100MB as I mentioned.

The two lists used approximately the same memory in my test with Python 2.6.6 
on Windows. An implementation detail such as this is likely to vary between 
interpreters across versions and platforms, including Jython and IronPython. A 
classic implementation detail is caching of small objects that occur 
frequently, such as small strings, integers less than 512, etc. For example:

In [1]: a = 20 * '5'
In [2]: b = 20 * '5'
In [3]: a is b
Out[3]: True

In [4]: a = 21 * '5'
In [5]: b = 21 * '5'
In [6]: a is b
Out[6]: False

It's best not to depend on this behavior.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Memory Usage of Strings

[Terry Reedy]

On 3/16/2011 3:51 PM, Santoso Wijaya wrote:

??

Python 2.7.1 (r271:86832, Nov 27 2010, 17:19:03) [MSC v.1500 64 bit
(AMD64)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> import sys
 >>> L = []
 >>> for i in xrange(10):
... L.append(str(i) * (1000 / len(str(i
...
 >>> sys.getsizeof(L)
824464


This is only the size of the list object and does not include the sum of 
sizes of the string objects. With 8-byth pointers, 824464 == 8*10 + 
(small bit of overhead) + extra space (for list to grow without 
reallocation and copy)



 >>> L = []
 >>> for i in xrange(2):
... L.append(str(i) * (5000 / len(str(i
...
 >>> sys.getsizeof(L)
178024


== 8*2 + extra

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

tkFileDialog Question

[garyr]

tkFileDialog.askdirectory() allows the selection of a directory. In my code
it displays a line of text at the top of the frame ("Please choose a
directory, then select OK"). A little below that the current path
("C:\Documents and Settings\Owner\My Documents\Python\...") is displayed as
a string and immediately below that in a small frame the basename of the
path is displayed. In my case the string showing the path is too long and is
displayed on two lines, the bottom part of which is obscured by the frame
displaying the basename. The title text referred to above can be changed by
means of the title option in the function call but there doesn't appear to
be an option for the path. Is there some other way the display of the path
or the frame displaying the basename could be eliminated or moved slightly?



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.path.walk() to get full path of all files

[dude]

awesome, that worked.  I'm not sure how the magic is working with your 
underscores there, but it's doing what I need.  thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.path.walk() to get full path of all files

[Benjamin Kaplan]

On Wed, Mar 16, 2011 at 5:00 PM, dude  wrote:
> awesome, that worked.  I'm not sure how the magic is working with your 
> underscores there, but it's doing what I need.  thanks.
> --

It's not magic at all. _ is just a variable name. When someone names a
variable _, it's just to let you know that they won't actually be
using that variable. In this case, os.walk returns three things: the
root, the list of directories, and the list of files. You only need
two of those for your code, so you ignore the third.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.path.walk() to get full path of all files

[Alexander Kapps]

On 16.03.2011 22:00, dude wrote:

awesome, that worked.  I'm not sure how the magic is working with your 
underscores there, but it's doing what I need.  thanks.


The underscore is no magic here. It's just a conventional variable 
name, saying "I m unused". One could also write:


for root, UNUSED, filenames in os.walk(root_path):
   ...


BTW. Please quote enough of the posts you reply to. Most people 
access this list/newsgroup per mail client or usenet reader, not per 
a webpage, so without quoting, they might not see the context of our 
post. Thank you.

--
http://mail.python.org/mailman/listinfo/python-list

Re: PostgreSQL vs MySQL

[Ben Finney]

a...@pythoncraft.com (Aahz) writes:

> >(I always recommend people to use PostgreSQL, though; which is
> >superior in almost every way, especially the C client library and the
> >wire protocol.)
>
> Can you point at a reference for the latter?  I have been trying to
> convince my company that PG is better than MySQL.

These may be helpful:

http://wiki.postgresql.org/wiki/Why_PostgreSQL_Instead_of_MySQL_2009>
http://www.wikivs.com/wiki/MySQL_vs_PostgreSQL>

-- 
 \  “I tell you the truth: this generation will certainly not pass |
  `\   away until all these things [the end of the world] have |
_o__)  happened.” —Jesus, c. 30 CE, as quoted in Matthew 24:34 |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Memory Usage of Strings

[Dan Stromberg]

On Wed, Mar 16, 2011 at 8:38 AM, Amit Dev  wrote:

> I'm observing a strange memory usage pattern with strings. Consider
> the following session. Idea is to create a list which holds some
> strings so that cumulative characters in the list is 100MB.
>
> >>> l = []
> >>> for i in xrange(10):
> ...  l.append(str(i) * (1000/len(str(i
>
> This uses around 100MB of memory as expected and 'del l' will clear that.
>
>
> >>> for i in xrange(2):
> ...  l.append(str(i) * (5000/len(str(i
>
> This is using 165MB of memory. I really don't understand where the
> additional memory usage is coming from.
>
> If I reduce the string size, it remains high till it reaches around
> 1000. In that case it is back to 100MB usage.
>
> Python 2.6.4 on FreeBSD.
>
> Regards,
> Amit
> --
> http://mail.python.org/mailman/listinfo/python-list
>

On Python 2.6.6 on Ubuntu 10.10:

$ cat pmu
#!/usr/bin/python

import os
import sys

list_ = []

if sys.argv[1] == '--first':
for i in xrange(10):
list_.append(str(i) * (1000/len(str(i
elif sys.argv[1] == '--second':
for i in xrange(2):
list_.append(str(i) * (5000/len(str(i
else:
sys.stderr.write('%s: Illegal sys.argv[1]\n' % sys.argv[0])
sys.exit(1)

os.system("ps aux | egrep '\<%d\>|^USER\>'" % os.getpid())

dstromberg-laptop-dstromberg:~/src/python-mem-use i686-pc-linux-gnu 10916 -
above cmd done 2011 Wed Mar 16 02:38 PM

$ make
./pmu --first
USER   PID %CPU %MEMVSZ   RSS TTY  STAT START   TIME COMMAND
1000 11063  0.0  3.4 110212 104436 pts/5   S+   14:38   0:00
/usr/bin/python ./pmu --first
1000 11064  0.0  0.0   1896   512 pts/5S+   14:38   0:00 sh -c ps
aux | egrep '\<11063\>|^USER\>'
1000 11066  0.0  0.0   4012   740 pts/5S+   14:38   0:00 egrep
\<11063\>|^USER\>
./pmu --second
USER   PID %CPU %MEMVSZ   RSS TTY  STAT START   TIME COMMAND
1000 11067 13.0  3.3 107540 101536 pts/5   S+   14:38   0:00
/usr/bin/python ./pmu --second
1000 11068  0.0  0.0   1896   508 pts/5S+   14:38   0:00 sh -c ps
aux | egrep '\<11067\>|^USER\>'
1000 11070  0.0  0.0   4012   740 pts/5S+   14:38   0:00 egrep
\<11067\>|^USER\>
dstromberg-laptop-dstromberg:~/src/python-mem-use i686-pc-linux-gnu 10916 -
above cmd done 2011 Wed Mar 16 02:38 PM

So on Python 2.6.6 + Ubuntu 10.10, the second is actually a little smaller
than the first.

Some issues you might ponder:
1) Does FreeBSD's malloc/free know how to free unused memory pages in the
middle of the heap (using mmap games), or does it only sbrk() down when the
end of the heap becomes unused, or does it never sbrk() back down at all?
I've heard various *ix's fall into one of these 3 groups in releasing unused
pages.

2) It mijght be just an issue of how frequently the interpreter garbage
collects; you could try adjusting this; check out the gc module.  Note that
it's often faster not to collect at every conceivable opportunity, but this
tends to add up the bytes pretty quickly in some scripts - for a while,
until the next collection.  So your memory use pattern will often end up
looking like a bit of a sawtooth function.

3) If you need strict memory use guarantees, you might be better off with a
language that's closer to the metal, like C - something that isn't garbage
collected is one parameter to consider.  If you already have something in
CPython, then Cython might help; Cython allows you to use C datastructures
from a dialect of Python.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: organizing many python scripts, in a large corporate environment.

[bukzor]

On Mar 16, 7:42 am, booklover  wrote:
> > I'm going to try to get our solution open-sourced, then I'll get your
> > feedback on it.
>
> Thanks bukzor! I think that it would be very helpful to have a library
> like this available.
>
> In the longer term, what do people think about the possibility of
> writing up a PEP to fix this problem in the core language? Anyone have
> any ideas on the cleanest way to achieve this goal?

If we had relative imports for scripts as well as package modules,
that would be the end of it.
When the relative import escapes the current package (or module) start
using filesystem semantics, where another dot means a parent
directory, and something that looks like a package is just a
directory. This replaces the "relative import from non-package" error,
so backward-compatibility is a non-issue.

bin/parrot/speak.py:
   def say(text):
   print text

bin/parrot/mccaw.py
#this is a script, not a module!
from ..speak import say

bin/guy/__init__.py
from ..parrot.speak import say


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: organizing many python scripts, in a large corporate environment.

[bukzor]

On Mar 15, 11:57 pm, "eryksun ()"  wrote:
> On Tuesday, March 15, 2011 12:44:48 AM UTC-4, bukzor wrote:
>
> > Currently it requires either: 1) no symlinks to scripts or 2)
> > installation of the pathtools to site-packages.
>
> An executable with a unique name on the system PATH could be executed it as a 
> subprocess that pipes the configured base directory back via stdout. Then you 
> can just do site.addsitedir(base).

I'm not able to picture this. Could you add detail?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: organizing many python scripts, in a large corporate environment.

[bukzor]

On Mar 16, 3:29 pm, bukzor  wrote:
> On Mar 15, 11:57 pm, "eryksun ()"  wrote:
>
> > On Tuesday, March 15, 2011 12:44:48 AM UTC-4, bukzor wrote:
>
> > > Currently it requires either: 1) no symlinks to scripts or 2)
> > > installation of the pathtools to site-packages.
>
> > An executable with a unique name on the system PATH could be executed it as 
> > a subprocess that pipes the configured base directory back via stdout. Then 
> > you can just do site.addsitedir(base).
>
> I'm not able to picture this. Could you add detail?

I finally understand. You mean something along the lines of `kde-
config`: an executable to help figure out the configuration at
runtime. This requires either installation or control of the $PATH
environment variable to work well, which makes not so useful to me.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: organizing many python scripts, in a large corporate environment.

[Dan Stromberg]

If you just want to be unhappy about the current situation, I hereby
formally bestow upon you permission to be so.  :)

The problem seems to be that you want to hook into the python process,
without hooking into the python process.  Were this possible, it seems we
might have a serious security issue to face.

Most sites don't have these difficulties, because they either allow changing
an environment variable globally, or allow changing site-packages globally.
Or more likely, they allow both.

virtualenv might be a good way for you to go, but I'm not confident about
that.  You'd still need to hook into that somehow.

You could perhaps push through python running /usr/local/mumble if it has
the same owner as the python interpreter or something, but that may have
security implications that'd need to be carefully considered.  Similarly,
you could perhaps push through python running ~/.python if it has
appropriate ownership, but it sounds like this wouldn't be viable in your
environment either.

The easiest thing to do, given your unusual constraints, is probably to put
the following at the top of each script:

#!/usr/bin/python

# start initializer boilerplate
import sys
sys.path.insert('/my/magic/directory')
import initializer_mod
# end initializer boilerplate

The vast majority of your site specifics would go in
/my/magic/directory/initializer_mod.py.  Then if you someday need to change
/my/magic/directory to /your/mundane/directory, sed -i is your friend, so
long as you keep these 3 (or 5) magic lines nice and consistent.  If you
really, truly, honest-to-whatever despise boilerplate, you could even set up
a script that automatically adds these lines at the beginning of each of
your scripts, if they aren't already there - in fact, it could replace
everything in between those two magic comments, so you can expand on that as
"needed".

But seriously - is it really so bad talking to your IT group about one small
change that could be used by multiple departments?  If the constraints
you're under outside of your IT group are truly as ossified as you make them
sound, shouldn't that add weight to your case when you present it to your
IT?

If you just want to get a handle on what is importing what in a modestly
sized directory (even a few hundred files isn't really that much), you could
probably try a graphviz graph generated using something like snakefood.

BTW, foo.py isn't generally a good thing to put on your $PATH:
http://lintian.debian.org/tags/script-with-language-extension.html
I see it as language vanity that needlessly complicates rewriting a script
from one language to another - it seems to come mostly from Windows which
has fits sometimes without file extensions, and the perl world where
language extensions on CGI scripts were seen as a means of advocacy.  What
goes on your $PATH is part of an interface though - there's little point in
clunking that up interfaces with implementation detail.  Yes, you might
actually want to rewrite your Python or Perl or Ruby into C or Go or Haskell
or OCaML someday.

Despite a degree of snarkiness in this post, I do hope this message helps
you somehow.

On Fri, Mar 11, 2011 at 9:25 PM, bukzor  wrote:

> We've been doing a fair amount of Python scripting, and now we have a
> directory with almost a hundred loosely related scripts. It's
> obviously time to organize this, but there's a problem. These scripts
> import freely from each other and although code reuse is  generally a
> good thing it makes it quite complicated to organize them into
> directories.
>
> There's a few things that you should know about our corporate
> environment:
>
> 1) I don't have access to the users' environment. Editing the
> PYTHONPATH is out, unless it happens in the script itself.
> 2) Users don't install things. Systems are expected to be *already*
> installed and working, so setup.py is not a solution.
>
> I'm quite willing to edit my import statements and do some minor
> refactoring, but the solutions I see currently require me to divide
> all the code strictly between "user runnable scripts" and "libraries",
> which isn't feasible, considering the amount of code.
>
> Has anyone out there solved a similar problem? Are you happy with it?
> --Buck
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: alphanumeric list

[not1xor1 (Alessandro)]

Il 15/03/2011 09:10, yqyq22 ha scritto:


I would like to put an alphanumeric string like this one
EE472A86441AF2E629DE360 in a list, then iterate inside the entire
string lenght and change each digit with a random digit.
Do u have some suggestion? thanks a lot


hi

I'm not an experienced python programmer but the following code should 
work:


# you need to import the random module to get random numbers
import random

# convert the string in a list (as string are read-only)
l = list('EE472A86441AF2E629DE360')
# initialize the random generator
random.seed()
# index
i = 0
# loop through the characters in the list
for c in l:
# if the current character is a digit
if c.isdigit():
# get a random integer in the 0-9 range
# convert it to a string and replace the old value
l[i] = str(random.randint(0,9))
# increment the list index
i +=1
# convert the modified list back to string and display it
print(''.join(l))

--
bye
!(!1|1)
--
http://mail.python.org/mailman/listinfo/python-list

creating RAW sockets

[moijes12]

Hi

I am unable to create RAW sockets using python.

Traceback (most recent call last):
  File "getsockopt_handler.py", line 6, in ?
send = socket(AF_INET,SOCK_RAW,IPPROTO_IP)
socket.error: (94, 'Socket type not supported')

I've tried changing the type from IPPROTO_IP to IPPROTO_RAW but the I
get the below error

Traceback (most recent call last):
  File "getsockopt_handler.py", line 7, in ?
send.bind((gethostbyname(gethostname()),5))
socket.error: (99, 'Cannot assign requested address')

I am running this on python 2.2 .Do I need to change any system
settings or do I need to use a newer python version.Also,please
suggest further reading.

thanks
moijes12
-- 
http://mail.python.org/mailman/listinfo/python-list

Fitting polynomial curve

[Astan Chee]

Hi,
I have 2 points in 3D space and a bunch of points in-between them. I'm
trying to fit a polynomial curve on it. Currently I'm looking through numpy
but I don't think the function exists to fit a function like this:
y = ax**4 + bx**3 + cx**2 + dx + e
(I'm not sure what thats called but one degree up from a cubic curve)
Also, I'm sure it'll take alot of time to brute force it like this but I'm
sure I'm missing something for this.
Thanks for any advice.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Fitting polynomial curve

[Terry Reedy]

On 3/17/2011 1:42 AM, Astan Chee wrote:

Hi,
I have 2 points in 3D space and a bunch of points in-between them. I'm
trying to fit a polynomial curve on it. Currently I'm looking through
numpy but I don't think the function exists to fit a function like this:
y = ax**4 + bx**3 + cx**2 + dx + e
(I'm not sure what thats called but one degree up from a cubic curve)


quartic


Also, I'm sure it'll take alot of time to brute force it like this but
I'm sure I'm missing something for this.


Look at scipy.

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Memory Usage of Strings

[Amit Dev]

Thanks Dan for the detailed reply. I suspect it is related to FreeBSD
malloc/free as you suggested. Here is the output of running your
script:

[16-bsd01 ~/work]$ python strm.py --first
USERPID %CPU %MEM   VSZ   RSS  TT  STAT STARTED  TIME COMMAND
amdev  6899  3.0  6.9 111944 107560  p0  S+9:57PM   0:01.20 python
strm.py --first (python2.5)
amdev  6900  0.0  0.1  3508  1424  p0  S+9:57PM   0:00.02 sh -c ps
aux | egrep '\\<6899\\>|^USER\\>'
amdev  6902  0.0  0.1  3380  1188  p0  S+9:57PM   0:00.01 egrep
\\<6899\\>|^USER\\>

[16-bsd01 ~/work]$ python strm.py --second
USERPID %CPU %MEM   VSZ   RSS  TT  STAT STARTED  TIME COMMAND
amdev  6903  0.0 10.5 166216 163992  p0  S+9:57PM   0:00.92 python
strm.py --second (python2.5)
amdev  6904  0.0  0.1  3508  1424  p0  S+9:57PM   0:00.02 sh -c ps
aux | egrep '\\<6903\\>|^USER\\>'
amdev  6906  0.0  0.1  3508  1424  p0  R+9:57PM   0:00.00 egrep
\\<6903\\>|^USER\\> (sh)

Regards,
Amit

On Thu, Mar 17, 2011 at 3:21 AM, Dan Stromberg  wrote:
>
> On Wed, Mar 16, 2011 at 8:38 AM, Amit Dev  wrote:
>>
>> I'm observing a strange memory usage pattern with strings. Consider
>> the following session. Idea is to create a list which holds some
>> strings so that cumulative characters in the list is 100MB.
>>
>> >>> l = []
>> >>> for i in xrange(10):
>> ...  l.append(str(i) * (1000/len(str(i
>>
>> This uses around 100MB of memory as expected and 'del l' will clear that.
>>
>>
>> >>> for i in xrange(2):
>> ...  l.append(str(i) * (5000/len(str(i
>>
>> This is using 165MB of memory. I really don't understand where the
>> additional memory usage is coming from.
>>
>> If I reduce the string size, it remains high till it reaches around
>> 1000. In that case it is back to 100MB usage.
>>
>> Python 2.6.4 on FreeBSD.
>>
>> Regards,
>> Amit
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>
> On Python 2.6.6 on Ubuntu 10.10:
>
> $ cat pmu
> #!/usr/bin/python
>
> import os
> import sys
>
> list_ = []
>
> if sys.argv[1] == '--first':
>     for i in xrange(10):
>     list_.append(str(i) * (1000/len(str(i
> elif sys.argv[1] == '--second':
>     for i in xrange(2):
>     list_.append(str(i) * (5000/len(str(i
> else:
>     sys.stderr.write('%s: Illegal sys.argv[1]\n' % sys.argv[0])
>     sys.exit(1)
>
> os.system("ps aux | egrep '\<%d\>|^USER\>'" % os.getpid())
>
> dstromberg-laptop-dstromberg:~/src/python-mem-use i686-pc-linux-gnu 10916 -
> above cmd done 2011 Wed Mar 16 02:38 PM
>
> $ make
> ./pmu --first
> USER   PID %CPU %MEM    VSZ   RSS TTY  STAT START   TIME COMMAND
> 1000 11063  0.0  3.4 110212 104436 pts/5   S+   14:38   0:00
> /usr/bin/python ./pmu --first
> 1000 11064  0.0  0.0   1896   512 pts/5    S+   14:38   0:00 sh -c ps
> aux | egrep '\<11063\>|^USER\>'
> 1000 11066  0.0  0.0   4012   740 pts/5    S+   14:38   0:00 egrep
> \<11063\>|^USER\>
> ./pmu --second
> USER   PID %CPU %MEM    VSZ   RSS TTY  STAT START   TIME COMMAND
> 1000 11067 13.0  3.3 107540 101536 pts/5   S+   14:38   0:00
> /usr/bin/python ./pmu --second
> 1000 11068  0.0  0.0   1896   508 pts/5    S+   14:38   0:00 sh -c ps
> aux | egrep '\<11067\>|^USER\>'
> 1000 11070  0.0  0.0   4012   740 pts/5    S+   14:38   0:00 egrep
> \<11067\>|^USER\>
> dstromberg-laptop-dstromberg:~/src/python-mem-use i686-pc-linux-gnu 10916 -
> above cmd done 2011 Wed Mar 16 02:38 PM
>
> So on Python 2.6.6 + Ubuntu 10.10, the second is actually a little smaller
> than the first.
>
> Some issues you might ponder:
> 1) Does FreeBSD's malloc/free know how to free unused memory pages in the
> middle of the heap (using mmap games), or does it only sbrk() down when the
> end of the heap becomes unused, or does it never sbrk() back down at all?
> I've heard various *ix's fall into one of these 3 groups in releasing unused
> pages.
>
> 2) It mijght be just an issue of how frequently the interpreter garbage
> collects; you could try adjusting this; check out the gc module.  Note that
> it's often faster not to collect at every conceivable opportunity, but this
> tends to add up the bytes pretty quickly in some scripts - for a while,
> until the next collection.  So your memory use pattern will often end up
> looking like a bit of a sawtooth function.
>
> 3) If you need strict memory use guarantees, you might be better off with a
> language that's closer to the metal, like C - something that isn't garbage
> collected is one parameter to consider.  If you already have something in
> CPython, then Cython might help; Cython allows you to use C datastructures
> from a dialect of Python.
>
>
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: creating RAW sockets

[Chris Rebert]

On Wed, Mar 16, 2011 at 10:36 PM, moijes12  wrote:
> Hi
>
> I am unable to create RAW sockets using python.
>
> Traceback (most recent call last):
>  File "getsockopt_handler.py", line 6, in ?
>    send = socket(AF_INET,SOCK_RAW,IPPROTO_IP)
> socket.error: (94, 'Socket type not supported')
>
> I've tried changing the type from IPPROTO_IP to IPPROTO_RAW but the I
> get the below error
>
> Traceback (most recent call last):
>  File "getsockopt_handler.py", line 7, in ?
>    send.bind((gethostbyname(gethostname()),5))
> socket.error: (99, 'Cannot assign requested address')
>
> I am running this on python 2.2 .Do I need to change any system
> settings or do I need to use a newer python version.

What operating system are you using?

Cheers,
Chris
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: creating RAW sockets

[Nobody]

On Wed, 16 Mar 2011 22:36:07 -0700, moijes12 wrote:

> Traceback (most recent call last):
>   File "getsockopt_handler.py", line 7, in ?
> send.bind((gethostbyname(gethostname()),5))
> socket.error: (99, 'Cannot assign requested address')

Specifying a port number isn't meaningful for a raw socket. At the kernel
level, the sin_port field in a sockaddr_in structure is used for the IP
protocol (e.g. 6 for TCP), if it's used at all.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: creating RAW sockets

[moijes12]

On Mar 17, 11:14 am, Nobody  wrote:
> On Wed, 16 Mar 2011 22:36:07 -0700, moijes12 wrote:
> > Traceback (most recent call last):
> >   File "getsockopt_handler.py", line 7, in ?
> >     send.bind((gethostbyname(gethostname()),5))
> > socket.error: (99, 'Cannot assign requested address')
>
> Specifying a port number isn't meaningful for a raw socket. At the kernel
> level, the sin_port field in a sockaddr_in structure is used for the IP
> protocol (e.g. 6 for TCP), if it's used at all.

@ Chris :
I am using windows xp sp2.

@ Nobody :
My main aim here is decode the IP header.

I tried the below code and it seems to work.I have shifted to python
2.5 (though I think it was more of a code problem).My main problem is
to decode the IP header

s = socket(AF_INET,SOCK_RAW,IPPROTO_IP)
r = socket(AF_INET,SOCK_RAW,IPPROTO_IP)
r.bind(('',0))
s.connect(('localhost',0))

for i in range(10) :
s.send(str(i))
data = r.recvfrom(256)
hdr = r.getsockopt(SOL_IP, IP_OPTIONS, 20)
print binascii.hexlify(hdr)

r.close()
s.close()

Here it prints nothing.I'll try some more stuff and will post my
findings in about 30 minutes.

thanks
moijes
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: creating RAW sockets

[moijes12]

On Mar 17, 11:28 am, moijes12  wrote:
> On Mar 17, 11:14 am, Nobody  wrote:
>
> > On Wed, 16 Mar 2011 22:36:07 -0700, moijes12 wrote:
> > > Traceback (most recent call last):
> > >   File "getsockopt_handler.py", line 7, in ?
> > >     send.bind((gethostbyname(gethostname()),5))
> > > socket.error: (99, 'Cannot assign requested address')
>
> > Specifying a port number isn't meaningful for a raw socket. At the kernel
> > level, the sin_port field in a sockaddr_in structure is used for the IP
> > protocol (e.g. 6 for TCP), if it's used at all.
>
> @ Chris :
> I am using windows xp sp2.
>
> @ Nobody :
> My main aim here is decode the IP header.
>
> I tried the below code and it seems to work.I have shifted to python
> 2.5 (though I think it was more of a code problem).My main problem is
> to decode the IP header
>
> s = socket(AF_INET,SOCK_RAW,IPPROTO_IP)
> r = socket(AF_INET,SOCK_RAW,IPPROTO_IP)
> r.bind(('',0))
> s.connect(('localhost',0))
>
> for i in range(10) :
>     s.send(str(i))
>     data = r.recvfrom(256)
>     hdr = r.getsockopt(SOL_IP, IP_OPTIONS, 20)
>     print binascii.hexlify(hdr)
>
> r.close()
> s.close()
>
> Here it prints nothing.I'll try some more stuff and will post my
> findings in about 30 minutes.
>
> thanks
> moijes

Hi

I tried the below code on python 3.0.1.This was an available example
in the manual and it was successfull in printing all packets

import socket

# the public network interface
HOST = socket.gethostbyname(socket.gethostname())

# create a raw socket and bind it to the public interface
s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP)
s.bind((HOST, 0))

# Include IP headers
s.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)

# receive all packages
s.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)

# receive a package
while 1 :
print(s.recvfrom(65565))

Now,please can someone guide(as in what should I read and NOT as in
give me the code) me in decoding the IP header of packets using python
3.0.1.

thanks
moijes
-- 
http://mail.python.org/mailman/listinfo/python-list