Re: Glob in python which supports the ** wildcard

[Simon Brunning]

On 22 November 2010 21:43, Martin Lundberg  wrote:
> Hi,
>
> I want to be able to let the user enter paths like this:
>
> apps/name/**/*.js
>
> and then find all the matching files in apps/name and all its
> subdirectories. However I found out that Python's glob function
> doesn't support the recursive ** wildcard. Is there any 3rd party glob
> function which do support **?

This does roughly what you want:

http://code.activestate.com/recipes/499305-locating-files-throughout-a-directory-tree/

-- 
Cheers,
Simon B.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Glob in python which supports the ** wildcard

[Simon Brunning]

On 23 November 2010 09:26, Martin Lundberg  wrote:
> It does not seem to support the ** wildcard? It will recursively seek
> for files matching a pattern like *.js but it won't support
> /var/name/**/*.js as root, will it?

I did say roughly. ;-) You'd need to do:

for filename in locate("*.js", "/var/name/"):
print filename

Adding support for ** is left as an exercise for the reader.

-- 
Cheers,
Simon B.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Reading bz2 file into numpy array

[Peter Otten]

Nobody wrote:

> On Mon, 22 Nov 2010 11:37:22 +0100, Peter Otten wrote:
> 
>>> is there a convenient way to read bz2 files into a numpy array?
>> 
>> Try
> 
>> f = bz2.BZ2File(filename)
>> data = numpy.fromstring(f.read(), numpy.float32)
> 
> That's going to hurt if the file is large.

Yes, but memory usage will peak at about 2*sizeof(data), and most scripts
need more data than just a single numpy.array.
In short: the OP is unlikely to run into the problem.

> You might be better off either extracting to a temporary file, or creating
> a pipe with numpy.fromfile() reading the pipe and either a thread or
> subprocess decompressing the data into the pipe.
 
I like to keep it simple, so if available RAM turns out to be the limiting 
factor I think extracting the data into a temporary file is a good backup 
plan. 

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python recursively __getattribute__

[bruno.desthuilli...@gmail.com]

On 22 nov, 21:44, Roman Dolgiy  wrote:
>> http://stackoverflow.com/questions/4247036/python-recursively-getattr...
>
> I need to support a lot of legacy code, with THC4k's approach I'll
> have to modify project's existing code to use obj.attr1.val instead of
> obj.attr1 but this is not suitable.

You should probably re-read THC4k's answer. His code works just fine
AFAICT:

"""
# the proxy maps attribute access to another object
class GetattrProxy(object):
def __init__(self, proxied, prefix=None):
self.proxied = proxied
self.prefix = prefix

def __getattr__(self, key):
attr = (key if self.prefix is None else self.prefix + '__' +
key)
try:
# if the proxied object has the attr return it
return getattr(self.proxied, attr)
except AttributeError:
# else just return another proxy
return GetattrProxy(self.proxied, attr)


# the thing you want to wrap
class Target(object):
attr1__attr2__attr3 = 5
attr2 = "attr2"


t = Target()
proxy = GetattrProxy(t)

print "proxy.attr1.attr2.attr3 : '%s'" % proxy.attr1.attr2.attr3
print "proxy.attr2 : '%s'" % proxy.attr2
"""
-- 
http://mail.python.org/mailman/listinfo/python-list

Making module content available in parent module

[Ulrich Eckhardt]

Hi!

Note up front: I'm using Python2.6 still, I guess with 2.7 test discovery, I
could get better results easier, right?

Now, my problem is I have a directory containing test scripts which I all
want to run. I used to run them individually and manually, but want to
avoid this overhead in the future.

 tests/
   foo.py # defines TestFoo1 and TestFoo2
   bar.py # defines TestBar1 and TestBar2

What I would like to do now is this:

 from tests import *
 unittest.main()

In other words, import all test files and run them. This does import them,
but it turns out that I end up with modules foo and bar, and the unittests
inside those are not found.

Am I approaching the test loading the wrong way?


Cheers!

Uli


PS: I've been trying a few things here, and stumbled across another thing
that could provide a solution. I can "from tests import *", but then all
these modules will pollute my namespace. I can "import tests", but then
neither of the submodules will be in "tests". I tried "import tests.*", but
that doesn't work. Is there no way to import a whole package but with its
namespace?

-- 
Domino Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Making module content available in parent module

[Gregor Horvath]

Hi,

On Tue, 23 Nov 2010 11:36:05 +0100
Ulrich Eckhardt  wrote:

> Now, my problem is I have a directory containing test scripts which I
> all want to run. I used to run them individually and manually, but
> want to avoid this overhead in the future.
> 
>  tests/
>foo.py # defines TestFoo1 and TestFoo2
>bar.py # defines TestBar1 and TestBar2

Nose does what you want:

http://packages.python.org/nose/

--
Gregor
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Making module content available in parent module

[Steven D'Aprano]

On Tue, 23 Nov 2010 11:36:05 +0100, Ulrich Eckhardt wrote:


>  tests/
>foo.py # defines TestFoo1 and TestFoo2 
>bar.py # defines TestBar1 and TestBar2
> 
> What I would like to do now is this:
> 
>  from tests import *
>  unittest.main()
> 
> In other words, import all test files and run them. This does import
> them, but it turns out that I end up with modules foo and bar, and the
> unittests inside those are not found.

Given the directory structure you show, I find that hard to believe. You 
should get an ImportError, as there is no module or package called 
"tests".

But suppose you turn tests into a proper package:

tests/
  __init__.py
  foo.py
  bar.py

You could have __init__.py include these lines:

from foo import *
from bar import *


Then later, when you do this:

from tests import * 

it will pick up everything from foo and bar, and unittest.main() should 
run those tests as well. I think.

Or you could just do:

for module in (foo, bar):
try:
unittest.main(module)
except SystemExit:
pass



> PS: I've been trying a few things here, and stumbled across another
> thing that could provide a solution. I can "from tests import *", but
> then all these modules will pollute my namespace. I can "import tests", 
> but then neither of the submodules will be in "tests". I tried "import
> tests.*", but that doesn't work. Is there no way to import a whole
> package but with its namespace?

The package needs to know what submodules to make available. Put inside 
__init__.py:


import foo
import bar



and then from outside the package, do this:

import tests

Now tests.foo and tests.bar will exist.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unittests with different parameters

[Jonathan Hartley]

On Nov 22, 11:38 am, Ulrich Eckhardt 
wrote:
> Hi!
>
> I'm writing tests and I'm wondering how to achieve a few things most
> elegantly with Python's unittest module.
>
> Let's say I have two flags invert X and invert Y. Now, for testing these, I
> would write one test for each combination. What I have in the test case is
> something like this:
>
>   def test_invert_flags(self):
>       """test flags to invert coordinates"""
>       tests = [((10, 20), INVERT_NONE, (10, 20)),
>                ((10, 20), INVERT_X, (-10, 20)),
>                ((10, 20), INVERT_Y, (10, -20))]
>       for input, flags, expected in tests:
>           res = do_invert(input, flags)
>           self.assertEqual(res, expected,
>                            "%s caused wrong results" % (flags,))
>
> So, what I do that I test the function 'do_invert' for different input
> combinations and verify the result. The ugly thing is that this will abort
> the whole test if one of the tests in the loop fails. So, my question is
> how do I avoid this?
>
> I know that I could write a common test function instead:
>
>   def _test_invert_flags(self, input, flags, expected):
>       res = do_invert(input, flags)
>       self.assertEqual(res, expected)
>
>   def test_invert_flags_non(self):
>       """test not inverting coordinates"""
>       self._test_invert_flags((10, 20), INVERT_NONE, (10, 20))
>
>   def test_invert_flags_x(self):
>       """test inverting X coordinates"""
>       self._test_invert_flags((10, 20), INVERT_X, (-10, 20))
>
>   def test_invert_flags_y(self):
>       """test inverting Y coordinates"""
>       self._test_invert_flags((10, 20), INVERT_Y, (10, -20))
>
> What I don't like here is that this is unnecessarily verbose and that it
> basically repeats information. Also, I'd rather construct the error message
> from the data instead of maintaining it in different places, because
> manually keeping those in sync is another, errorprone burden.
>
> Any suggestions?
>
> Uli
>
> --
> Domino Laser GmbH
> Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932


The following is a bit ghastly, I'm not sure I'd recommend it, but if
you are determined, you could try dynamically adding test methods to
the test class. The following is untested - I suspect I have made a
schoolboy error in attempting to make methods out of functions - but
something like it might work:


class MyTestClass(unittest.TestCase):
  pass

testdata = [
  (INPUTS, EXPECTED),
  (INPUTS, EXPECTED),
  (INPUTS, EXPECTED),
]

for index, (input, expected) in enumerate(testdata):
# the following sets an attribute on MyTestClass
# the names of the attributes are 'test_1', 'test_2', etc
# the value of the attributes is a test method that performs the
assert
setattr(
MyTestClass,
'test_%d' % (index,),
lambda s: s.assertEquals(METHOD_UNDER_TEST(*input), expected)
)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need advices regarding the strings (str, unicode, coding) used as interface for an external library.

[jmfauth]

Thanks for the reply. Subject closed.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python recursively __getattribute__

[Roman Dolgiy]

Thanks to Andreas Waldenburger, THC4k (http://stackoverflow.com/
questions/4247036/python-recursively-getattribute) and others for
their tips. I was able to find solution:

class Null(object):
def __repr__(self):
return ""

def __str__(self):
return ''

def __nonzero__(self):
return False


class ResultAttrFactory(type):
_cache = {}

@classmethod
def prepare(cls, base, result):
dict_ = ResultAttr.__dict__.copy()
dict_.update({
'_ResultAttr__base': base,
'_ResultAttr__result': result})
return ('ResultAttr', (base,), dict_)

def __new__(cls, base, result):
if (base, result) in cls._cache:
type_ = cls._cache[(base, result)]
else:
type_ = super(ResultAttrFactory, cls).__new__(
cls, *cls.prepare(base, result))
cls._cache[(base, result)] = type_
return type_

def __init__(cls, base, result):
pass


class ResultAttr:
"""Should be used only with ResultAttrFactory"""
@staticmethod
def __new__(cls, arg1, name):
return cls.__base.__new__(cls, arg1)

def __init__(self, arg1, name):
self.__name = name
super(self.__class__, self).__init__(arg1)

def get_result_attr(self, name):
if self.__result.is_denorm_attr(name):
attr = getattr(self.__result, name, None)
else:
attr = getattr(self.__result, name)
return attr

def __getattr__(self, name):
lookup_name = "%s__%s" % (self.__name, name)
attr = self.get_result_attr(lookup_name)
if type(attr).__name__ == 'ResultAttr':
type_ = attr.__base
elif attr is None:
type_ = Null
else:
type_ = type(attr)
result_attr = ResultAttrFactory(
type_, self.__result)(attr, lookup_name)
return result_attr


class BaseResult(object):
"""
>>> class Result(BaseResult):
... def __init__(self, *args, **kwargs):
... self.x = 35
... self.y = 5
... self.y__a = 3
... self.y__b = 'hello'
... self.y__c__x = [1, 2, 3]
... super(Result, self, *args, **kwargs)
>>> r = Result()
>>> r.x
35
>>> type(r.x)

>>> r.y
5
>>> type(r.y)  # doctest:+ELLIPSIS

>>> r.y.a
3
>>> r.y.b
'hello'
>>> r.y.c  # Is there any way to raise AtrributeError here?

>>> r.y.d
Traceback (most recent call last):
AttributeError: 'Result' object has no attribute 'y__d'
>>> r.y.c.x
[1, 2, 3]
"""
def is_denorm_attr(self, name):
return bool([k for k in self.__dict__.keys() if "%s__" % name
in k])

def __getattribute__(self, name):
attr = super(BaseResult, self).__getattribute__(name)
if name in ('__dict__', 'is_denorm_attr'):
return attr
if self.is_denorm_attr(name):
return ResultAttrFactory(type(attr), self)(attr, name)
else:
return attr


if __name__ == '__main__':
import doctest
doctest.testmod()

https://gist.github.com/710977

The only problem is:

>>> r.y.c


Is there any way to raise AtrributeError here?

Also is there any negative side effects of such method?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Scheme as a virtual machine?

[Keith H Duggar]

On Nov 22, 5:12 pm, Raffael Cavallaro
 wrote:
> On 2010-11-22 11:25:34 -0500, scattered said:
>
> > And you don't think that [JH] could write a book about Haskell
> > if he honestly came to think that it were a superior all-aroung
> > language?
>
> Until he actually does, he has a financial interest in trash-talking
> Haskell. This makes anything he says about Haskell suspect.
>
> >  The fact that he *didn't* mindlessly reject [musical note lang] in favor of
> > [Irish Ship Of The Desert] when [musical note lang] came out (despite
> > the fact that at the time his company
> > was deeply (exclusively?) invested in [Irish Ship Of The Desert] and
> > arguably had a vested
> > interest in having [musical note lang] fail to gain support) suggests
> > that he is able
> > to fairly evaluate the merits of other languages.
>
> No, it suggests that he saw that supporting the Irish Ship Of The
> Desert meant going up against Microsoft, so he jumped to the MS
> supported variant of the Donut Dromedary.
>
> You miss the fundamental point; having a financial interest in the
> outcome of a debate makes anything that person says an advertisement
> for his financial interests, not a fair assessment.

There is a well-known name for such illogical reasoning: ad hominem.
When a person poses an /argument/, nothing personal outside of the
/argument/ is relevant. Thus, your claim that "anything that person
says ..." is not only obvious hyperbole it is also illogical.

It is a common refuge of those who cannot support their position
with fact and logic. On more than one occasion Jon Harrop has all
but crushed Ertugrul in this very forum with /source code/; that
is as objective as it gets.

KHD
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Scheme as a virtual machine?

[Raffael Cavallaro]

On 2010-11-23 10:08:12 -0500, Keith H Duggar said:


There is a well-known name for such illogical reasoning: ad hominem.


You don't understand ad hominem:

"The ad hominem is a classic logical fallacy,[2] but it is not always 
fallacious. For in some instances, questions of personal conduct, 
character, motives, etc., are legitimate and relevant to the issue.[3]"


Source: 

Sometimes the person's conduct and motives *are relevant* to the point 
under discussion. Financial conflict of interest is a perfect example 
where it *is* legitimate and relevant to explore a person's motives and 
conduct outside of the debate.


In this case, JH's conduct outside of the debate (i.e., the fact that 
he earns his living by selling tools and training for a particular set 
of languages) and his motives (i.e., he is therefore financially 
motivated to present these languages in the best possible light and to 
trash-talk other languages), render his arguments in the debate 
inherently suspect.


warmest regards,

Ralph

--
Raffael Cavallaro

--
http://mail.python.org/mailman/listinfo/python-list

Re: Making module content available in parent module

[Ulrich Eckhardt]

Steven D'Aprano wrote:
> On Tue, 23 Nov 2010 11:36:05 +0100, Ulrich Eckhardt wrote:
>> PS: I've been trying a few things here, and stumbled across another
>> thing that could provide a solution. I can "from tests import *", but
>> then all these modules will pollute my namespace. I can "import tests",
>> but then neither of the submodules will be in "tests". I tried "import
>> tests.*", but that doesn't work. Is there no way to import a whole
>> package but with its namespace?
> 
> The package needs to know what submodules to make available. Put inside
> __init__.py:
> 
> 
> import foo
> import bar
> 
> and then from outside the package, do this:
> 
> import tests
> 
> Now tests.foo and tests.bar will exist.

I've been reading up on packages, but I didn't find anything like that in
the documentation, all I found was the meaning of __all__. If I import the
modules explicitly there, there's no need to define __all__, unless there
are some I don't want to import there, right? Well, I'll try it and see. ;)

Steven, thank you for this explanation!

Uli

-- 
Domino Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Scheme as a virtual machine?

[Keith H Duggar]

On Nov 23, 10:34 am, Raffael Cavallaro
 wrote:
> On 2010-11-23 10:08:12 -0500, Keith H Duggar said:
> > On Nov 22, 5:12 pm, Raffael Cavallaro 
> >  wrote:
> > > On 2010-11-22 11:25:34 -0500, scattered said:
> > >
> > > > And you don't think that [JH] could write a book about Haskell
> > > > if he honestly came to think that it were a superior all-aroung
> > > > language?
> > >
> > > Until he actually does, he has a financial interest in trash-talking
> > > Haskell. This makes anything he says about Haskell suspect.
> > >
> > > >  The fact that he *didn't* mindlessly reject [musical note lang] in 
> > > > favor of
> > > > [Irish Ship Of The Desert] when [musical note lang] came out (despite
> > > > the fact that at the time his company
> > > > was deeply (exclusively?) invested in [Irish Ship Of The Desert] and
> > > > arguably had a vested
> > > > interest in having [musical note lang] fail to gain support) suggests
> > > > that he is able
> > > > to fairly evaluate the merits of other languages.
> > >
> > > No, it suggests that he saw that supporting the Irish Ship Of The
> > > Desert meant going up against Microsoft, so he jumped to the MS
> > > supported variant of the Donut Dromedary.
> > >
> > > You miss the fundamental point; having a financial interest in the
> > > outcome of a debate makes anything that person says an advertisement
> > > for his financial interests, not a fair assessment.
> >
> > There is a well-known name for such illogical reasoning: ad hominem.
> > When a person poses an /argument/, nothing personal outside of the
> > /argument/ is relevant. Thus, your claim that "anything that person
> > says ..." is not only obvious hyperbole it is also illogical.
> >
> > It is a common refuge of those who cannot support their position
> > with fact and logic. On more than one occasion Jon Harrop has all
> > but crushed Ertugrul in this very forum with /source code/; that
> > is as objective as it gets.
>
> You don't understand ad hominem:
>
> "The ad hominem is a classic logical fallacy,[2] but it is not always
> fallacious. For in some instances, questions of personal conduct,
> character, motives, etc., are legitimate and relevant to the issue.[3]"
>
> Source: 
>
> Sometimes the person's conduct and motives *are relevant* to the point
> under discussion. Financial conflict of interest is a perfect example
> where it *is* legitimate and relevant to explore a person's motives and
> conduct outside of the debate.
>
> In this case, JH's conduct outside of the debate (i.e., the fact that
> he earns his living by selling tools and training for a particular set
> of languages) and his motives (i.e., he is therefore financially
> motivated to present these languages in the best possible light and to
> trash-talk other languages), render his arguments in the debate
> inherently suspect.

You don't understand the implications of your own words:

   "having a financial interest in the outcome of a debate makes
   anything that person says an advertisement for his financial
   interests, not a fair assessment."

is substantially different from

   "render his arguments in the debate inherently suspect."

Do you understand how? Hint, see my comment regarding "hyperbole"
and also consider the relationship between the qualifier "anything"
and universal quantification.

I think if you think a bit more carefully you will come to see how
your original statement was indeed fallacious ad hominem. (And that
specific example remains so regardless of which common approach to
informal logic you take ie whether you choose one that is more or
less sympathetic to ad hominem in general.)

KHD
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Scheme as a virtual machine?

[D'Arcy J.M. Cain]

On Tue, 23 Nov 2010 10:34:22 -0500
Raffael Cavallaro 
wrote:
> On 2010-11-23 10:08:12 -0500, Keith H Duggar said:
> > There is a well-known name for such illogical reasoning: ad hominem.
> You don't understand ad hominem:

Perhaps you don't understand it.

> "The ad hominem is a classic logical fallacy,[2] but it is not always 
> fallacious. For in some instances, questions of personal conduct, 
> character, motives, etc., are legitimate and relevant to the issue.[3]"

So, explain how motive makes the logic wrong in this case then.

> In this case, JH's conduct outside of the debate (i.e., the fact that 
> he earns his living by selling tools and training for a particular set 
> of languages) and his motives (i.e., he is therefore financially 
> motivated to present these languages in the best possible light and to 
> trash-talk other languages), render his arguments in the debate 
> inherently suspect.

Fine.  Suspect his arguments to the point that you examine them closely
and then explain what you found erroneous in them.  Don't just claim
that we should dismiss them because of who made them.

You know, it's just possible that Jon actually investigated Haskell
before choosing to focus on CL.  That would make his opinion carry more
weight, not less.

Remind me, how is this relevant to Python?

-- 
D'Arcy J.M. Cain  |  Democracy is three wolves
http://www.druid.net/darcy/|  and a sheep voting on
+1 416 425 1212 (DoD#0082)(eNTP)   |  what's for dinner.
-- 
http://mail.python.org/mailman/listinfo/python-list

Time and date operation

[huisky]

Hi everyone,

Could you anybody shed lights to me? Say I have two dics.
>>> cstart
defaultdict(, {15424: ['Dec', '6', '18:57:40'], 552:
['Dec', '7', '09:31:00'], 15500: ['Dec', '6', '20:17:02'], 18863:
['Dec', '7', '13:14:47'], 18291: ['Dec', '6', '21:01:17'], 18969:
['Dec', '7', '14:28:42'], 18937: ['Dec', '7', '14:21:34']})
>>> ccompl
defaultdict(, {15424: ['Dec', '6', '19:42:55'], 18291:
['Dec', '6', '21:01:28'], 15500: ['Dec', '6', '20:26:03'], 18863:
['Dec', '7', '13:24:07']})

and I need to calculate the difference time if the key value is the
same in both dics.

thanks in advance

Huisky
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Time and date operation

[Chris Rebert]

On Tue, Nov 23, 2010 at 9:47 AM, huisky  wrote:
> Hi everyone,
>
> Could you anybody shed lights to me? Say I have two dics.
 cstart
> defaultdict(, {15424: ['Dec', '6', '18:57:40'], 552:
> ['Dec', '7', '09:31:00'], 15500: ['Dec', '6', '20:17:02'], 18863:
> ['Dec', '7', '13:14:47'], 18291: ['Dec', '6', '21:01:17'], 18969:
> ['Dec', '7', '14:28:42'], 18937: ['Dec', '7', '14:21:34']})
 ccompl
> defaultdict(, {15424: ['Dec', '6', '19:42:55'], 18291:
> ['Dec', '6', '21:01:28'], 15500: ['Dec', '6', '20:26:03'], 18863:
> ['Dec', '7', '13:24:07']})
>
> and I need to calculate the difference time if the key value is the
> same in both dics.

Create datetime.datetime objects. Subtract one from another and you'll
get a datetime.timedelta object representing the difference between
them.
The fine manual: http://docs.python.org/library/datetime.html#datetime-objects

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Time and date operation

[Chris Rebert]

> -Original Message-
> From: c...@rebertia.com [mailto: c...@rebertia.com]
> Sent: 2010年11月23日 19:12
> To: huisky
> Cc: python-list@python.org
> Subject: Re: Time and date operation
>
> On Tue, Nov 23, 2010 at 9:47 AM, huisky  wrote:
>> Hi everyone,
>>
>> Could you anybody shed lights to me? Say I have two dics.
> cstart
>> defaultdict(, {15424: ['Dec', '6', '18:57:40'], 552:
>> ['Dec', '7', '09:31:00'], 15500: ['Dec', '6', '20:17:02'], 18863:
>> ['Dec', '7', '13:14:47'], 18291: ['Dec', '6', '21:01:17'], 18969:
>> ['Dec', '7', '14:28:42'], 18937: ['Dec', '7', '14:21:34']})
> ccompl
>> defaultdict(, {15424: ['Dec', '6', '19:42:55'], 18291:
>> ['Dec', '6', '21:01:28'], 15500: ['Dec', '6', '20:26:03'], 18863:
>> ['Dec', '7', '13:24:07']})
>>
>> and I need to calculate the difference time if the key value is the
>> same in both dics.
>
> Create datetime.datetime objects. Subtract one from another and you'll
> get a datetime.timedelta object representing the difference between
> them.
> The fine manual: http://docs.python.org/library/datetime.html#datetime-objects
>
On Tue, Nov 23, 2010 at 10:42 AM, Huisky  wrote:
> Hi,
>
> Thanks. I'm wondering is datetime.datetime objects able to read 'Dec', for 
> example?

Yes. Use the %b format directive when calling datetime.datetime.strptime().
http://docs.python.org/library/datetime.html#datetime.datetime.strptime
http://docs.python.org/library/time.html#time.strftime

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Subprocess Call works on Windows, but not Ubuntu

[Brett Bowman]

I ran into an interesting problem trying to spawn a subprocess, so I thought
I'd ask if the experts could explain it to me.  I'm spawning a subprocess to
run "pdf2txt.py", which is a tool that is distributed with PDFminer to do
moderately advanced text-dumps of PDFs.  Yet when I run the same code on my
two dev machines - one Win XP, the other Ubuntu 10.04 or 10.10 - it only
works on the former and not the later. And its not terribly complicated
code.

# Code Start
sp_line = 'python pdf2txt.py -p 1 -o %s "%s"' % ('temp.out', pdf_filename)
print sp_line
sp = subprocess.Popen(sp_line)
sp.wait()
with open('temp.out', 'r') as pdf_handle:
#Do stuff to read the file

The output from the print statements reads:
python pdf2txt.py -p 1 -o temp.out "Aarts et al (2009).pdf"

That command works on both systems when copied directly to the command-line,
and the python script it is a part of works on the Windows machine, but I
can't the script to work on Ubuntu for the life of me.  What am I missing?

/b/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Subprocess Call works on Windows, but not Ubuntu

[Chris Rebert]

On Tue, Nov 23, 2010 at 11:28 AM, Brett Bowman  wrote:
> I ran into an interesting problem trying to spawn a subprocess, so I thought
> I'd ask if the experts could explain it to me.  I'm spawning a subprocess to
> run "pdf2txt.py", which is a tool that is distributed with PDFminer to do
> moderately advanced text-dumps of PDFs.  Yet when I run the same code on my
> two dev machines - one Win XP, the other Ubuntu 10.04 or 10.10 - it only
> works on the former and not the later. And its not terribly complicated
> code.
> # Code Start
> sp_line = 'python pdf2txt.py -p 1 -o %s "%s"' % ('temp.out', pdf_filename)
> print sp_line
> sp = subprocess.Popen(sp_line)

> python pdf2txt.py -p 1 -o temp.out "Aarts et al (2009).pdf"
> That command works on both systems when copied directly to the command-line,
> and the python script it is a part of works on the Windows machine, but I
> can't the script to work on Ubuntu for the life of me.  What am I missing?

Quoting the docs (for the Nth time; emphasis added):
"""
On Unix, with shell=False (default): args should normally be a
sequence. ***If a string is specified for args***, it will be used as
the name or path of the program to execute; ***this will only work if
the program is being given no arguments.***
""" http://docs.python.org/library/subprocess.html#subprocess.Popen

Fixed version:
sp_args = ['python', 'pdf2txt.py', '-p', '1', '-o', 'temp.out', pdf_filename]
sp = subprocess.Popen(sp_args)

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Collect output to string

[Burton Samograd]

Hello,

I was wondering if there is any way in python to 'collect output to
string' as in some lisps/schemes.  Output being, printed output to the
console using print.

Thanks.

--
Burton Samograd

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Collect output to string

[Chris Rebert]

On Tue, Nov 23, 2010 at 11:53 AM, Burton Samograd  wrote:
> Hello,
>
> I was wondering if there is any way in python to 'collect output to
> string' as in some lisps/schemes.  Output being, printed output to the
> console using print.

Rebind sys.stdout to a StringIO object.
http://docs.python.org/library/sys.html#sys.stdout
http://docs.python.org/library/stringio.html

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Subprocess Call works on Windows, but not Ubuntu

[Brett Bowman]

Ah, that fixed it.  Thank you.

On Tue, Nov 23, 2010 at 11:37 AM, Chris Rebert  wrote:

> On Tue, Nov 23, 2010 at 11:28 AM, Brett Bowman  wrote:
> > I ran into an interesting problem trying to spawn a subprocess, so I
> thought
> > I'd ask if the experts could explain it to me.  I'm spawning a subprocess
> to
> > run "pdf2txt.py", which is a tool that is distributed with PDFminer to do
> > moderately advanced text-dumps of PDFs.  Yet when I run the same code on
> my
> > two dev machines - one Win XP, the other Ubuntu 10.04 or 10.10 - it only
> > works on the former and not the later. And its not terribly complicated
> > code.
> > # Code Start
> > sp_line = 'python pdf2txt.py -p 1 -o %s "%s"' % ('temp.out',
> pdf_filename)
> > print sp_line
> > sp = subprocess.Popen(sp_line)
> 
> > python pdf2txt.py -p 1 -o temp.out "Aarts et al (2009).pdf"
> > That command works on both systems when copied directly to the
> command-line,
> > and the python script it is a part of works on the Windows machine, but I
> > can't the script to work on Ubuntu for the life of me.  What am I
> missing?
>
> Quoting the docs (for the Nth time; emphasis added):
> """
> On Unix, with shell=False (default): args should normally be a
> sequence. ***If a string is specified for args***, it will be used as
> the name or path of the program to execute; ***this will only work if
> the program is being given no arguments.***
> """ http://docs.python.org/library/subprocess.html#subprocess.Popen
>
> Fixed version:
> sp_args = ['python', 'pdf2txt.py', '-p', '1', '-o', 'temp.out',
> pdf_filename]
> sp = subprocess.Popen(sp_args)
>
> Cheers,
> Chris
> --
> http://blog.rebertia.com
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Collect output to string

[Burton Samograd]

Chris Rebert  writes:

> On Tue, Nov 23, 2010 at 11:53 AM, Burton Samograd  wrote:
>> Hello,
>>
>> I was wondering if there is any way in python to 'collect output to
>> string' as in some lisps/schemes.  Output being, printed output to the
>> console using print.
>
> Rebind sys.stdout to a StringIO object.
> http://docs.python.org/library/sys.html#sys.stdout
> http://docs.python.org/library/stringio.html

Thanks for the tip.  Here's my function:

def with_output_to_string(f, args):
 oldstdout = sys.stdout
 buffer = StringIO.StringIO()
 sys.stdout = buffer
 apply(f, args)
 sys.stdout = oldstdout
 return buffer.getvalue()

Any way I could improve it?

--
Burton Samograd

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Collect output to string

[MRAB]

On 23/11/2010 20:59, Burton Samograd wrote:

Chris Rebert  writes:


On Tue, Nov 23, 2010 at 11:53 AM, Burton Samograd  wrote:

Hello,

I was wondering if there is any way in python to 'collect output to
string' as in some lisps/schemes.  Output being, printed output to the
console using print.


Rebind sys.stdout to a StringIO object.
http://docs.python.org/library/sys.html#sys.stdout
http://docs.python.org/library/stringio.html


Thanks for the tip.  Here's my function:

def with_output_to_string(f, args):
  oldstdout = sys.stdout
  buffer = StringIO.StringIO()
  sys.stdout = buffer
  apply(f, args)
  sys.stdout = oldstdout
  return buffer.getvalue()

Any way I could improve it?


Use a try...finally block in an exception occurs:

try:
   apply(f, args)
finally:
sys.stdout = oldstdout

and possibly also replace the older:

apply(f, args)

with the newer:

f(*args)

You might also want to handle keyword arguments.
--
http://mail.python.org/mailman/listinfo/python-list

progamming approach review, child processes

[astar]

So I have not done much with child processes before.

I have an input of programs to be updated, a child process that does
the
compiles and links (with a log output to an individual file), and a
process wait
at the end.  Except the child process can hang (at the moment, the
problem that might show up is: waiting for
input).  At which point a convenience  becomes a  hinderance.

As it happens the idea of doing the compiles in parallel is supported
by the
ports system I am working with.  So I thought I could avoid the wait
syntax, do
a lot of children, save all the process hmm I am not sure what to call
the
pointers,  and after a while interrogate the process information to
see if all
the children got done.  At some point, I could think a child is hung
and maybe
do some useful reporting.  Realize that this might be the next day
because the
compiles might take a while.

Is this a reasonable approach?

Thank you for your consideration.

Possibly pointless information:

openbsd 4.8 workstation, packages, but using the port system, want
only to
compile packages I have installed.  There is a utility to tell me
which packages
are out of date.  And I really know  I could use pkg_add -u.  I did
some
attachments of the programs, but I suppose they will get scrubbed..

python is I think 2.6.5?



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Collect output to string

[Ian]

On Nov 23, 1:59 pm, Burton Samograd  wrote:
> Thanks for the tip.  Here's my function:
>
> def with_output_to_string(f, args):
>      oldstdout = sys.stdout
>      buffer = StringIO.StringIO()
>      sys.stdout = buffer
>      apply(f, args)
>      sys.stdout = oldstdout
>      return buffer.getvalue()
>
> Any way I could improve it?

You should wrap the inner function call in a try-finally call to
ensure that the old stdout gets restored even if f raises an
exception.

Also, the `apply` function is deprecated.  Use `f(*args)` instead.

The function as a whole would be a bit more Pythonic if written as a
decorator IMO, but that's your call.

Cheers,
Ian
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Collect output to string

[Chris Rebert]

On Tue, Nov 23, 2010 at 1:15 PM, Ian  wrote:
> On Nov 23, 1:59 pm, Burton Samograd  wrote:
>> Thanks for the tip.  Here's my function:
>>
>> def with_output_to_string(f, args):
>>      oldstdout = sys.stdout
>>      buffer = StringIO.StringIO()
>>      sys.stdout = buffer
>>      apply(f, args)
>>      sys.stdout = oldstdout
>>      return buffer.getvalue()
>>
>> Any way I could improve it?
>
> You should wrap the inner function call in a try-finally call to
> ensure that the old stdout gets restored even if f raises an
> exception.
>
> Also, the `apply` function is deprecated.  Use `f(*args)` instead.
>
> The function as a whole would be a bit more Pythonic if written as a
> decorator IMO, but that's your call.

A context manager could also be worth considering.
http://docs.python.org/library/stdtypes.html#context-manager-types

Cheers,
Chris
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Collect output to string

[Terry Reedy]

On 11/23/2010 3:02 PM, Chris Rebert wrote:

On Tue, Nov 23, 2010 at 11:53 AM, Burton Samograd  wrote:

Hello,

I was wondering if there is any way in python to 'collect output to
string' as in some lisps/schemes.  Output being, printed output to the
console using print.


Rebind sys.stdout to a StringIO object.
http://docs.python.org/library/sys.html#sys.stdout
http://docs.python.org/library/stringio.html


If you are using print or print(), you can redirect output to the 
StringIO object with >>sfile or file=sfile. I use the latter in a custom 
test function where I normally want output to the screen but 
occasionally want to capture test reports.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Scheme as a virtual machine?

[Ertugrul Söylemez]

Keith H Duggar  wrote:

> It is a common refuge of those who cannot support their position with
> fact and logic. On more than one occasion Jon Harrop has all but
> crushed Ertugrul in this very forum with /source code/; that is as
> objective as it gets.

Since Jon has financial reasons to invest time doing this and I don't,
this is nowhere near "crushing" or "objective".  It's simply
meaningless.  If someone pays me for writing proof code or coming up
with challenges, then I will, and I assure you, I would give him a hard
time, since I'm an experienced Haskell programmer, who uses it for many
different, practical purposes in the real world outside of academia.

And I stated explicitly many times that (without being paid) I don't
feel like wasting time proving my point to Jon, who would just come up
with new arbitrary arguments and challenges anyway, as he does all the
time.  Jon doesn't and cannot acknowledge valid arguments, so it would
be an ongoing, pointless cycle.

After all, he was the only one posing stupid challenges on me at all,
deliberately constructing problems to be easy to solve in his languages.
When I would challenge him, the picture would change, but I think, this
is stupid and infantile enough not to do it.  In fact, I've even done it
once and proved my point that way (which, as always, he didn't
acknowledge, but I don't care anymore).


Greets,
Ertugrul


-- 
nightmare = unsafePerformIO (getWrongWife >>= sex)
http://ertes.de/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CGI FieldStorage instances?

[Gnarlodious]

On Nov 22, 11:32 pm, Dennis Lee Bieber wrote:

>         Or upgrade to some modernistic framework wherein the application is
> a monolithic program and the "name/" portion maps to methods/functions
> within the application...

Yes, that describes what I am looking for! Is there such a modernistic
framework? Links?

-- Gnarlie, K5ZN
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CGI FieldStorage instances?

[Ian Kelly]

On 11/23/2010 7:01 PM, Gnarlodious wrote:

On Nov 22, 11:32 pm, Dennis Lee Bieber wrote:


 Or upgrade to some modernistic framework wherein the application is
a monolithic program and the "name/" portion maps to methods/functions
within the application...


Yes, that describes what I am looking for! Is there such a modernistic
framework? Links?


Try Django[1] or TurboGears[2].

[1] http://www.djangoproject.com/
[2] http://www.turbogears.org/

Cheers,
Ian

--
http://mail.python.org/mailman/listinfo/python-list

Arrays

[Garland Fulton]

Is there a way I can define an Array of and unknown size so I can add and
remove to or from it?

Are arrays immutable?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Arrays

[Ian Kelly]

On 11/23/2010 10:55 PM, Garland Fulton wrote:

Is there a way I can define an Array of and unknown size so I can add
and remove to or from it?


Do you mean the arrays of the array module, or NumPy arrays, or 
something else entirely?  In the first case, yes; arrays behave more or 
less like lists, but more efficiently and with type constraints.  In the 
second case, I believe you have to explicitly resize the array in order 
to add new elements to it.



Are arrays immutable?


No in either case.

Cheers,
Ian

--
http://mail.python.org/mailman/listinfo/python-list

Ensuring symmetry in difflib.SequenceMatcher

[John Yeung]

I'm generally pleased with difflib.SequenceMatcher:  It's probably not
the best available string matcher out there, but it's in the standard
library and I've seen worse in the wild.  One thing that kind of
bothers me is that it's sensitive to which argument you pick as "seq1"
and which you pick as "seq2":

Python 2.6.1 (r261:67517, Dec  4 2008, 16:51:00) [MSC v.1500 32 bit
(Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import difflib
>>> difflib.SequenceMatcher(None, 'BYRD', 'BRADY').ratio()
0.2
>>> difflib.SequenceMatcher(None, 'BRADY', 'BYRD').ratio()
0.3
>>>

Is this a bug?  I am guessing the algorithm is implemented correctly,
and that it's just an inherent property of the algorithm used.  It's
certainly not what I'd call a desirably property.  Are there any
simple adjustments that can be made without sacrificing (too much)
performance?

John
-- 
http://mail.python.org/mailman/listinfo/python-list