Re: [Python-Dev] PEP 420 - dynamic path computation is missing rationale

2012-05-23 Thread Eric V. Smith 
On 05/22/2012 09:49 PM, PJ Eby wrote:
> On Tue, May 22, 2012 at 8:40 PM, Eric V. Smith  > wrote:
> 
> On 5/22/2012 2:37 PM, Guido van Rossum wrote:
> > Okay, I've been convinced that keeping the dynamic path feature is a
> > good idea. I am really looking forward to seeing the rationale added
> > to the PEP -- that's pretty much the last thing on my list that made
> > me hesitate. I'll leave the details of exactly how the parent path is
> > referenced up to the implementation team (several good points were
> > made), as long as the restriction that sys.path must be modified in
> > place is lifted.
> 
> I've updated the PEP. Let me know how it looks.
> 
> 
> My name is misspelled in it, but otherwise it looks fine.  ;-)

Oops, sorry. Fixed (I think).

> I have not updated the implementation yet. I'm not exactly sure how I'm
> going to convert from a path list of unknown origin to ('sys', 'path')
> or ('foo', '__path__'). I'll look at it later tonight to see if it's
> possible. I'm hoping it doesn't require major surgery to
> importlib._bootstrap.
> 
> 
> It shouldn't - all you should need is to use
> getattr(sys.modules[self.modname], self.attr) instead of referencing a
> parent path object directly.

The problem isn't the lookup, it's coming up with self.modname and
self.attr. As it currently stands, PathFinder.find_module is given the
parent path, not the module name and attribute name used to look up the
parent path using sys.modules and getattr.

Eric.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 420 - dynamic path computation is missing rationale

2012-05-23 Thread Nick Coghlan 
On Wed, May 23, 2012 at 10:31 PM, Eric V. Smith  wrote:
> On 05/22/2012 09:49 PM, PJ Eby wrote:
>> It shouldn't - all you should need is to use
>> getattr(sys.modules[self.modname], self.attr) instead of referencing a
>> parent path object directly.
>
> The problem isn't the lookup, it's coming up with self.modname and
> self.attr. As it currently stands, PathFinder.find_module is given the
> parent path, not the module name and attribute name used to look up the
> parent path using sys.modules and getattr.

Right, that's what PJE and I were discussing. Instead of passing in
the path object directly, you can instead pass an object that *lazily*
retrieves the path object in its __iter__ method:

class LazyIterable:
"""On iteration, retrieves a reference to a named iterable and
returns an iterator over that iterable"""
def __init__(self, modname, attribute):
self.modname = modname
self.attribute = attribute
def __iter__(self):
mod = import_module(self.modname) # Will almost always get
a hit directly in sys.modules
return iter(getattr(mod, self.attribute)

Where importlib currently passes None or sys.path as the path argument
to find_module(), instead pass "LazyIterable('sys', 'path')" and where
it currently passes package.__path__, instead pass
"LazyIterable(package.__name__, '__path__')".

The existing for loop iteration and tuple() calls should then take
care of the lazy lookup automatically.

That way, the only code that needs to know the values of modname and
attribute is the code that already has access to those values.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 420 - dynamic path computation is missing rationale

2012-05-23 Thread Eric V. Smith 
On 05/23/2012 09:02 AM, Nick Coghlan wrote:
> On Wed, May 23, 2012 at 10:31 PM, Eric V. Smith  wrote:
>> On 05/22/2012 09:49 PM, PJ Eby wrote:
>>> It shouldn't - all you should need is to use
>>> getattr(sys.modules[self.modname], self.attr) instead of referencing a
>>> parent path object directly.
>>
>> The problem isn't the lookup, it's coming up with self.modname and
>> self.attr. As it currently stands, PathFinder.find_module is given the
>> parent path, not the module name and attribute name used to look up the
>> parent path using sys.modules and getattr.
> 
> Right, that's what PJE and I were discussing. Instead of passing in
> the path object directly, you can instead pass an object that *lazily*
> retrieves the path object in its __iter__ method:

Hey, one message at a time! I'm just reading those now.

I'd like to hear Brett's comments on this approach.

Eric.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 420 - dynamic path computation is missing rationale

2012-05-23 Thread PJ Eby 
On May 23, 2012 9:02 AM, "Nick Coghlan"  wrote:
>
> On Wed, May 23, 2012 at 10:31 PM, Eric V. Smith 
wrote:
> > On 05/22/2012 09:49 PM, PJ Eby wrote:
> >> It shouldn't - all you should need is to use
> >> getattr(sys.modules[self.modname], self.attr) instead of referencing a
> >> parent path object directly.
> >
> > The problem isn't the lookup, it's coming up with self.modname and
> > self.attr. As it currently stands, PathFinder.find_module is given the
> > parent path, not the module name and attribute name used to look up the
> > parent path using sys.modules and getattr.
>
> Right, that's what PJE and I were discussing. Instead of passing in
> the path object directly, you can instead pass an object that *lazily*
> retrieves the path object in its __iter__ method:
>
>class LazyIterable:
>"""On iteration, retrieves a reference to a named iterable and
> returns an iterator over that iterable"""
>def __init__(self, modname, attribute):
>self.modname = modname
>self.attribute = attribute
>def __iter__(self):
>mod = import_module(self.modname) # Will almost always get
> a hit directly in sys.modules
>return iter(getattr(mod, self.attribute)
>
> Where importlib currently passes None or sys.path as the path argument
> to find_module(), instead pass "LazyIterable('sys', 'path')" and where
> it currently passes package.__path__, instead pass
> "LazyIterable(package.__name__, '__path__')".
>
> The existing for loop iteration and tuple() calls should then take
> care of the lazy lookup automatically.
>
> That way, the only code that needs to know the values of modname and
> attribute is the code that already has access to those values.

Perhaps calling it a ModulePath instead of a LazyIterable would be better?

Also, this is technically a change from PEP 302, which says the actual
sys.path or __path__ are passed to find_module().  I'm not sure whether any
find_module() code ever written actually *cares* about this, though.
(Especially if, as I believe I understand in this context, we're only
talking about meta-importers.)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Benchmark performance...

2012-05-23 Thread stefan brunthaler 
Hi,

as Antoine pointed out in the corresponding issue
(http://bugs.python.org/issue14757#msg160870), measuring/assessing
real-world performance of my patch would be interesting. I mentioned
that I am not aware of any relevant Python 3 program/application to
report numbers for (but guess that the speedups should persist.) Since
nobody came up with an answer yet, I figured it would be a good idea
to ask everybody on python-dev for suggestions...

Regards,
--stefan
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 420 - dynamic path computation is missing rationale

2012-05-23 Thread Brett Cannon 
On Wed, May 23, 2012 at 9:10 AM, Eric V. Smith  wrote:

> On 05/23/2012 09:02 AM, Nick Coghlan wrote:
> > On Wed, May 23, 2012 at 10:31 PM, Eric V. Smith 
> wrote:
> >> On 05/22/2012 09:49 PM, PJ Eby wrote:
> >>> It shouldn't - all you should need is to use
> >>> getattr(sys.modules[self.modname], self.attr) instead of referencing a
> >>> parent path object directly.
> >>
> >> The problem isn't the lookup, it's coming up with self.modname and
> >> self.attr. As it currently stands, PathFinder.find_module is given the
> >> parent path, not the module name and attribute name used to look up the
> >> parent path using sys.modules and getattr.
> >
> > Right, that's what PJE and I were discussing. Instead of passing in
> > the path object directly, you can instead pass an object that *lazily*
> > retrieves the path object in its __iter__ method:
>
> Hey, one message at a time! I'm just reading those now.
>
> I'd like to hear Brett's comments on this approach.


If I understand the proposal correctly, this would be a change in
NamespaceLoader in how it sets __path__ and in no way affect any other code
since __import__() just grabs the object on __path__ and passes as an
argument to the meta path finders which just iterate over the object, so I
have no objections to it.

-Brett
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 420 - dynamic path computation is missing rationale

2012-05-23 Thread PJ Eby 
On Wed, May 23, 2012 at 3:02 PM, Brett Cannon  wrote:

> If I understand the proposal correctly, this would be a change in
> NamespaceLoader in how it sets __path__ and in no way affect any other code
> since __import__() just grabs the object on __path__ and passes as an
> argument to the meta path finders which just iterate over the object, so I
> have no objections to it.


That's not *quite* the proposal (but almost).  The change would also mean
that __import__() instead passes a ModulePath (aka Nick's LazyIterable)
instance to the meta path finders, which just iterate over it.  But other
than that, yes.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 420 - dynamic path computation is missing rationale

2012-05-23 Thread Brett Cannon 
On Wed, May 23, 2012 at 3:35 PM, PJ Eby  wrote:

> On Wed, May 23, 2012 at 3:02 PM, Brett Cannon  wrote:
>
>> If I understand the proposal correctly, this would be a change in
>> NamespaceLoader in how it sets __path__ and in no way affect any other code
>> since __import__() just grabs the object on __path__ and passes as an
>> argument to the meta path finders which just iterate over the object, so I
>> have no objections to it.
>
>
> That's not *quite* the proposal (but almost).  The change would also mean
> that __import__() instead passes a ModulePath (aka Nick's LazyIterable)
> instance to the meta path finders, which just iterate over it.  But other
> than that, yes.
>

And why does __import__() need to construct that? I thought NamespaceLoader
was going to be making these "magical" __path__ objects that detected
changes and thus update themselves as necessary and just stick them on the
object. Why specifically does __import__() need to play a role?
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #14814: addition of the ipaddress module (stage 1 - code and tests)

2012-05-23 Thread Sandro Tosi 
On Tue, May 22, 2012 at 10:43 PM, Terry Reedy  wrote:
> On 5/22/2012 3:59 PM, Sandro Tosi wrote:
>> On Sun, May 20, 2012 at 7:18 PM, Terry Reedy  wrote
 +    Args:
 +        first: the first IPv4Address or IPv6Address in the range.
 +        last: the last IPv4Address or IPv6Address in the range.
 +
 +    Returns:
 +        An iterator of the summarized IPv(4|6) network objects.
>>>
>>>
>>> Very clear as to types.
>>
>>
>> I don't think I get exactly what you mean here.
>
>
> This docstring clearly says what the input type is instead of the more vague
> 'address'. Also, the output is pretty clearly an iterable of IPv#Address
> objects. I meant to contrast this as a good example compared to some of the
> previous docstrings.

Ah now I see, thanks for fixing my understanding ;)

Cheers,
-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 420 - dynamic path computation is missing rationale

2012-05-23 Thread Eric V. Smith 
On 05/23/2012 03:56 PM, Brett Cannon wrote:
> 
> 
> On Wed, May 23, 2012 at 3:35 PM, PJ Eby  > wrote:
> 
> On Wed, May 23, 2012 at 3:02 PM, Brett Cannon  > wrote:
> 
> If I understand the proposal correctly, this would be a change
> in NamespaceLoader in how it sets __path__ and in no way affect
> any other code since __import__() just grabs the object on
> __path__ and passes as an argument to the meta path finders
> which just iterate over the object, so I have no objections to it.
> 
> 
> That's not *quite* the proposal (but almost).  The change would also
> mean that __import__() instead passes a ModulePath (aka Nick's
> LazyIterable) instance to the meta path finders, which just iterate
> over it.  But other than that, yes.
> 
> 
> And why does __import__() need to construct that? I thought
> NamespaceLoader was going to be making these "magical" __path__ objects
> that detected changes and thus update themselves as necessary and just
> stick them on the object. Why specifically does __import__() need to
> play a role? 

Assume that we're talking about importing either a top-level namespace
package named 'parent' and a nested namespace package parent.child.

The problem is that NamespaceLoader is just passed the parent path
(typically sys.path, but if a sub-package then parent.__path__). The
concern is that if the parent path object is replaced:
sys.path = sys.path + ['new-dir']
or
parent.__path__ = ['new-dir']
then the NamespaceLoader instance can no longer detect changes to
parent_path.

So the proposed solution is for NamespaceLoader to be told the name of
the parent module ('sys' or 'parent') and the attribute name to use to
find the path ('path' or '__path__').

Here's another suggestion: instead of modifying the finder/loader code
to pass these names through, assume that we can always find
(module_name, attribute_name) with this code:

def find_parent_path_names(module):
parent, dot, me = module.__name__.rpartition('.')
if dot == '':
return 'sys', 'path'
return parent, '__path__'

>>> import glob
>>> find_parent_path_names(glob)
('sys', 'path')
>>> import unittest.test.test_case
>>> find_parent_path_names(unittest.test.test_case)
('unittest.test', '__path__')

I guess it's a little more fragile than passing in these names to
NamespaceLoader, but it requires less code to change.

I think I'll whip this up in the pep-420 branch.

Eric.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python db2 installation error

2012-05-23 Thread PremAnand Lakshmanan 
I want to install python db2 package for Python but Im unable to install it.

I have installed the easy_install and Im able to successfully import the
easy_install.

My easy_install location :c:/python27/lib/site-packages/

My db2 egg location c:/python27/ibm_db-1.0.5-py2.7-win32.egg

How would my installation command look like in the shell,

I tried this command and it gives me invalid error,

C:\Python27\Scripts>easy_install
c:/python27/lib/site-packages/ibm_db-1.0.5-py2.
7-win32.egg
error: Not a URL, existing file, or requirement spec:
'c:/python27/lib/site-pack
ages/ibm_db-1.0.5-py2.7-win32.egg'


-- 
Prem
408-393-2545
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python db2 installation error

2012-05-23 Thread Terry Reedy 

On 5/23/2012 7:00 PM, PremAnand Lakshmanan wrote:

I want to install python db2 package for Python but Im unable to install it.


pydev list is for development of future python releases. Ask questions 
about using existing python releases on python-list or the gmane mirror.


--
Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 420 - dynamic path computation is missing rationale

2012-05-23 Thread Eric V. Smith 
> Here's another suggestion: instead of modifying the finder/loader code
> to pass these names through, assume that we can always find
> (module_name, attribute_name) with this code:
> 
> def find_parent_path_names(module):
> parent, dot, me = module.__name__.rpartition('.')
> if dot == '':
> return 'sys', 'path'
> return parent, '__path__'
> 
 import glob
 find_parent_path_names(glob)
> ('sys', 'path')
 import unittest.test.test_case
 find_parent_path_names(unittest.test.test_case)
> ('unittest.test', '__path__')
> 
> I guess it's a little more fragile than passing in these names to
> NamespaceLoader, but it requires less code to change.
> 
> I think I'll whip this up in the pep-420 branch.

I tried this approach and it works fine. The only caveat is that it
assumes that the parent path can always be computed as described above,
independent of what's passed in to PathFinder.load_module(). I think
that's reasonable, since load_module() itself hard-codes sys.path if the
supplied path is missing.

I've checked this in to the pep-420 branch. I prefer this approach over
Nick's because it doesn't require any changes to any existing
interfaces. The changes are contained to the namespace package code and
don't affect other code in importlib.

Assuming this approach is acceptable, I'm done with the PEP except for
adding some examples.

And I'm done with the implementation except for adding tests and a few
small tweaks.

Eric.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 420 - dynamic path computation is missing rationale

2012-05-23 Thread PJ Eby 
On Wed, May 23, 2012 at 8:24 PM, Eric V. Smith  wrote:

> I tried this approach and it works fine. The only caveat is that it
> assumes that the parent path can always be computed as described above,
> independent of what's passed in to PathFinder.load_module(). I think
> that's reasonable, since load_module() itself hard-codes sys.path if the
> supplied path is missing.
>

Technically, PEP 302 says that finders aren't allowed to assume their
parent packages are imported:

""" However, the find_module() method isn't necessarily always called
during an actual import: meta tools that analyze import dependencies (such
as freeze, Installer or py2exe) don't actually load modules, so a finder
shouldn't *depend* on the parent package being available in sys.modules."""

OTOH, that's finders, and I think we're dealing with loaders here.
Splitting hairs, perhaps, but at least it's in a good cause.  ;-)


I've checked this in to the pep-420 branch. I prefer this approach over
> Nick's because it doesn't require any changes to any existing
> interfaces. The changes are contained to the namespace package code and
> don't affect other code in importlib.
>
> Assuming this approach is acceptable, I'm done with the PEP except for
> adding some examples.
>
> And I'm done with the implementation except for adding tests and a few
> small tweaks.
>

Yay!
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 420 - dynamic path computation is missing rationale

2012-05-23 Thread Eric V. Smith 
On 5/23/2012 8:58 PM, PJ Eby wrote:
> On Wed, May 23, 2012 at 8:24 PM, Eric V. Smith  > wrote:
> 
> I tried this approach and it works fine. The only caveat is that it
> assumes that the parent path can always be computed as described above,
> independent of what's passed in to PathFinder.load_module(). I think
> that's reasonable, since load_module() itself hard-codes sys.path if the
> supplied path is missing.
> 
> 
> Technically, PEP 302 says that finders aren't allowed to assume their
> parent packages are imported:
> 
> """ However, the find_module() method isn't necessarily always called
> during an actual import: meta tools that analyze import dependencies
> (such as freeze, Installer or py2exe) don't actually load modules, so a
> finder shouldn't /depend/ on the parent package being available in
> sys.modules."""
> 
> OTOH, that's finders, and I think we're dealing with loaders here. 
> Splitting hairs, perhaps, but at least it's in a good cause.  ;-)

I guess I could store the passed-in parent path, and use that if it
can't be found through sys.modules.

I'm not sure I can conjure up code to test this.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 420 - dynamic path computation is missing rationale

2012-05-23 Thread PJ Eby 
On Wed, May 23, 2012 at 9:02 PM, Eric V. Smith  wrote:

> On 5/23/2012 8:58 PM, PJ Eby wrote:
> > On Wed, May 23, 2012 at 8:24 PM, Eric V. Smith  > > wrote:
> >
> > I tried this approach and it works fine. The only caveat is that it
> > assumes that the parent path can always be computed as described
> above,
> > independent of what's passed in to PathFinder.load_module(). I think
> > that's reasonable, since load_module() itself hard-codes sys.path if
> the
> > supplied path is missing.
> >
> >
> > Technically, PEP 302 says that finders aren't allowed to assume their
> > parent packages are imported:
> >
> > """ However, the find_module() method isn't necessarily always called
> > during an actual import: meta tools that analyze import dependencies
> > (such as freeze, Installer or py2exe) don't actually load modules, so a
> > finder shouldn't /depend/ on the parent package being available in
> > sys.modules."""
> >
> > OTOH, that's finders, and I think we're dealing with loaders here.
> > Splitting hairs, perhaps, but at least it's in a good cause.  ;-)
>
> I guess I could store the passed-in parent path, and use that if it
> can't be found through sys.modules.
>
> I'm not sure I can conjure up code to test this.
>

I actually was suggesting that we change PEP 302, if it became an issue.
;-)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 420 - dynamic path computation is missing rationale

2012-05-23 Thread Nick Coghlan 
On Thu, May 24, 2012 at 11:02 AM, Eric V. Smith  wrote:
> On 5/23/2012 8:58 PM, PJ Eby wrote:
>> OTOH, that's finders, and I think we're dealing with loaders here.
>> Splitting hairs, perhaps, but at least it's in a good cause.  ;-)
>
> I guess I could store the passed-in parent path, and use that if it
> can't be found through sys.modules.
>
> I'm not sure I can conjure up code to test this.

I don't think there's a need to change anything from your current
strategy, but we should be clear in the docs:

1. Finders should *not* assume their parent packages have been loaded
(and should not load them implicitly)
2. Loaders *can* assume their parent packages have already been loaded
and are present in sys.modules (and can complain if they're not there)

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python db2 installation error

2012-05-23 Thread Georg Brandl 
Am 24.05.2012 01:45, schrieb Terry Reedy:
> On 5/23/2012 7:00 PM, PremAnand Lakshmanan wrote:
>> I want to install python db2 package for Python but Im unable to install it.
> 
> pydev list is for development of future python releases. Ask questions 
> about using existing python releases on python-list or the gmane mirror.

No please?

Georg

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com