Re: [Python-Dev] is the 'path' argument to an importer's find_module() just a hint?
Brett Cannon wrote: > I just discovered frozen packages set their __path__ simply to their > name and not to a list as expected (http://bugs.python.org/issue4211). > This made me think about the 'path' argument to find_module() and > whether it can be treated as simply a hint or should always be > seriously looked at. > > Take frozen modules, for instance. If the 'path' argument is meant to > always be considered then if a frozen module is within a package a > check should be done to make sure that the parent package is in 'path' > somewhere. But if it is simply a hint, then 'path' should be ignored > and whether the module can be found should depend fully on > imp.is_frozen(). > > So, what do people think? Should 'path' for find_module() always be > taken into consideration, or only when it happens to be convenient? >From PEP 302: importer.find_module(fullname, path=None) This method will be called with the fully qualified name of the module. If the importer is installed on sys.meta_path, it will receive a second argument, which is None for a top-level module, or package.__path__ for submodules or subpackages[7]. It should return a loader object if the module was found, or None if it wasn't. If find_module() raises an exception, it will be propagated to the caller, aborting the import. [7] The path argument to importer.find_module() is there because the pkg.__path__ variable may be needed at this point. It may either come from the actual parent module or be supplied by imp.find_module() or the proposed imp.get_loader() function. Note the first "may" in the footnote. If an importer doesn't need pkg.__path__ to find submodules then it isn't obliged to use it, but the import machinery provides it as a convenience. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] is the 'path' argument to an importer's find_module() just a hint?
On Mon, Oct 27, 2008 at 3:50 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote: > Brett Cannon wrote: >> I just discovered frozen packages set their __path__ simply to their >> name and not to a list as expected (http://bugs.python.org/issue4211). >> This made me think about the 'path' argument to find_module() and >> whether it can be treated as simply a hint or should always be >> seriously looked at. >> >> Take frozen modules, for instance. If the 'path' argument is meant to >> always be considered then if a frozen module is within a package a >> check should be done to make sure that the parent package is in 'path' >> somewhere. But if it is simply a hint, then 'path' should be ignored >> and whether the module can be found should depend fully on >> imp.is_frozen(). >> >> So, what do people think? Should 'path' for find_module() always be >> taken into consideration, or only when it happens to be convenient? > > From PEP 302: > > importer.find_module(fullname, path=None) > >This method will be called with the fully qualified name of the >module. If the importer is installed on sys.meta_path, it will >receive a second argument, which is None for a top-level module, or >package.__path__ for submodules or subpackages[7]. It should return >a loader object if the module was found, or None if it wasn't. If >find_module() raises an exception, it will be propagated to the >caller, aborting the import. > > > [7] The path argument to importer.find_module() is there because the >pkg.__path__ variable may be needed at this point. It may either >come from the actual parent module or be supplied by >imp.find_module() or the proposed imp.get_loader() function. > > > Note the first "may" in the footnote. If an importer doesn't need > pkg.__path__ to find submodules then it isn't obliged to use it, but the > import machinery provides it as a convenience. > Good enough for me. Then I am just going to ignore the 'path' argument for frozen modules but use it to short-circuit imports for built-in modules. -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] is the 'path' argument to an importer's find_module() just a hint?
On 07:24 pm, [EMAIL PROTECTED] wrote: On Mon, Oct 27, 2008 at 3:50 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote: Brett Cannon wrote: Good enough for me. Then I am just going to ignore the 'path' argument for frozen modules but use it to short-circuit imports for built-in modules. For what it's worth, Twisted uses the __path__ attribute to facilitate its plugin mechanism. If this changes so that importing from frozen packages no longer honors __path__, then Twisted will no longer support plugins if the package to be plugged into is frozen. What is the motivation to change this? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] is the 'path' argument to an importer's find_module() just a hint?
Brett Cannon schrieb: > I just discovered frozen packages set their __path__ simply to their > name and not to a list as expected (http://bugs.python.org/issue4211). > This made me think about the 'path' argument to find_module() and > whether it can be treated as simply a hint or should always be > seriously looked at. > > Take frozen modules, for instance. If the 'path' argument is meant to > always be considered then if a frozen module is within a package a > check should be done to make sure that the parent package is in 'path' > somewhere. But if it is simply a hint, then 'path' should be ignored > and whether the module can be found should depend fully on > imp.is_frozen(). > > So, what do people think? Should 'path' for find_module() always be > taken into consideration, or only when it happens to be convenient? At the moment I don't care about find_module for frozen modules/packages (is someone still using these?), but all the code I remember that manipulates a packages __path__ would most certainly break if it finds a string instead of a list. Thomas ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Why do we care about -OPT:Olimit=0?
Python's configure.in contains this check:
# disable check for icc since it seems to pass, but generates a warning
if test "$CC" = icc
then
ac_cv_opt_olimit_ok=no
fi
AC_MSG_CHECKING(whether $CC accepts -OPT:Olimit=0)
AC_CACHE_VAL(ac_cv_opt_olimit_ok,
[ac_save_cc="$CC"
CC="$CC -OPT:Olimit=0"
AC_TRY_RUN([int main() { return 0; }],
ac_cv_opt_olimit_ok=yes,
ac_cv_opt_olimit_ok=no,
ac_cv_opt_olimit_ok=no)
CC="$ac_save_cc"])
AC_MSG_RESULT($ac_cv_opt_olimit_ok)
Why do we care about that particular obscure compiler flag? I'm trying to
build Python 2.4.x with the Sun Studio 12 compiler, which appears to operate
in the same broken fashion as icc. It warns that it doesn't support the
option but still exits with a zero status which makes configure incorrectly
think it does support the flag.
This same chunk of code seems to be in configure.in dating back to 1997 with
this comment:
r8948 | guido | 1997-10-09 15:24:13 -0500 (Thu, 09 Oct 1997) | 2 lines
Don Beaudry's changes to support SGI_ABI on Irix 6.x.
Can this check be ripped out?
Skip
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] is the 'path' argument to an importer's find_module() just a hint?
On Mon, Oct 27, 2008 at 12:31 PM, <[EMAIL PROTECTED]> wrote: > On 07:24 pm, [EMAIL PROTECTED] wrote: >> >> On Mon, Oct 27, 2008 at 3:50 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote: >>> >>> Brett Cannon wrote: > >> Good enough for me. Then I am just going to ignore the 'path' argument >> for frozen modules but use it to short-circuit imports for built-in >> modules. > > For what it's worth, Twisted uses the __path__ attribute to facilitate its > plugin mechanism. > If this changes so that importing from frozen packages no > longer honors __path__, then Twisted will no longer support plugins if the > package to be plugged into is frozen. > But how do you manipulate it for frozen packages currently? Importing a frozen package currently has a bad __path__ value:: >>> import __phello__; __phello__.__path__ '__phello__' Notice how that is not a list. Do you special-case frozen packages? > What is the motivation to change this? Ease of implementation for me (this has not gone far enough to be an official change; as of this moment this is just for my import rewrite vaporware). imp.is_frozen() tells you whether a module is frozen or not, and thus whether to use imp.init_frozen(). So implementing a meta_path importer for frozen modules can be nothing more than calls to imp.is_frozen() and imp.init_frozen() if you just ignore the 'path' argument. -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] is the 'path' argument to an importer's find_module() just a hint?
On Mon, Oct 27, 2008 at 1:03 PM, Thomas Heller <[EMAIL PROTECTED]> wrote: > Brett Cannon schrieb: >> I just discovered frozen packages set their __path__ simply to their >> name and not to a list as expected (http://bugs.python.org/issue4211). >> This made me think about the 'path' argument to find_module() and >> whether it can be treated as simply a hint or should always be >> seriously looked at. >> >> Take frozen modules, for instance. If the 'path' argument is meant to >> always be considered then if a frozen module is within a package a >> check should be done to make sure that the parent package is in 'path' >> somewhere. But if it is simply a hint, then 'path' should be ignored >> and whether the module can be found should depend fully on >> imp.is_frozen(). >> >> So, what do people think? Should 'path' for find_module() always be >> taken into consideration, or only when it happens to be convenient? > > At the moment I don't care about find_module for frozen modules/packages > (is someone still using these?), Beats me. At this point I am only implementing them for backwards-compatibility, but they are exposing some inconsistencies in how various aspects of import are implemented. If I had my way they would be an optional thing that you would have to explicitly put on sys.meta_path to avoid the overhead. > but all the code I remember that > manipulates a packages __path__ would most certainly break if it > finds a string instead of a list. > Oh, definitely. That's why I filed the bug report on top of asking for feedback on this. -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why do we care about -OPT:Olimit=0?
On Mon, Oct 27, 2008 at 1:22 PM, <[EMAIL PROTECTED]> wrote:
> Python's configure.in contains this check:
>
># disable check for icc since it seems to pass, but generates a warning
>if test "$CC" = icc
>then
> ac_cv_opt_olimit_ok=no
>fi
>
>AC_MSG_CHECKING(whether $CC accepts -OPT:Olimit=0)
>AC_CACHE_VAL(ac_cv_opt_olimit_ok,
>[ac_save_cc="$CC"
>CC="$CC -OPT:Olimit=0"
>AC_TRY_RUN([int main() { return 0; }],
> ac_cv_opt_olimit_ok=yes,
> ac_cv_opt_olimit_ok=no,
> ac_cv_opt_olimit_ok=no)
>CC="$ac_save_cc"])
>AC_MSG_RESULT($ac_cv_opt_olimit_ok)
>
> Why do we care about that particular obscure compiler flag? I'm trying to
> build Python 2.4.x with the Sun Studio 12 compiler, which appears to operate
> in the same broken fashion as icc. It warns that it doesn't support the
> option but still exits with a zero status which makes configure incorrectly
> think it does support the flag.
>
> This same chunk of code seems to be in configure.in dating back to 1997 with
> this comment:
>
>r8948 | guido | 1997-10-09 15:24:13 -0500 (Thu, 09 Oct 1997) | 2 lines
>
>Don Beaudry's changes to support SGI_ABI on Irix 6.x.
>
> Can this check be ripped out?
Irix support has been removed in 3.0 so it can at least be tossed
there if the OS is the only reason to have the flag aroung.
-Brett
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] is the 'path' argument to an importer's find_module() just a hint?
On 08:24 pm, [EMAIL PROTECTED] wrote: On Mon, Oct 27, 2008 at 12:31 PM, <[EMAIL PROTECTED]> wrote: On 07:24 pm, [EMAIL PROTECTED] wrote: On Mon, Oct 27, 2008 at 3:50 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote: For what it's worth, Twisted uses the __path__ attribute to facilitate its plugin mechanism. But how do you manipulate it for frozen packages currently? Importing a frozen package currently has a bad __path__ value:: >>> import __phello__; __phello__.__path__ '__phello__' Notice how that is not a list. Do you special-case frozen packages? Oh. Well, in that case, it's already broken, nevermind :). I naively assumed that someone would have reported a bug in Twisted if it were different from i.e. zip importing. I suppose it makes sense - plugins are used for "twistd" subcommands, and it's unlikely that you'd want to freeze "twistd"; few applications use Twisted plugins themselves yet. I misunderstood your earlier description; I thought that this (the invalid __path__) was the desired change, not the current state of the world. (Of course if you could *change* frozen packages to have a valid __path__ I'd appreciate that...) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] is the 'path' argument to an importer's find_module() just a hint?
[EMAIL PROTECTED] wrote: > On 07:24 pm, [EMAIL PROTECTED] wrote: >> On Mon, Oct 27, 2008 at 3:50 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote: >>> Brett Cannon wrote: > >> Good enough for me. Then I am just going to ignore the 'path' argument >> for frozen modules but use it to short-circuit imports for built-in >> modules. > > For what it's worth, Twisted uses the __path__ attribute to facilitate > its plugin mechanism. If this changes so that importing from frozen > packages no longer honors __path__, then Twisted will no longer support > plugins if the package to be plugged into is frozen. > > What is the motivation to change this? If I had to guess, I'd say Brett found some time to tinker with his import_in_py implementation. That has been an interesting exercise in blowing dust out of some of the dark corners of CPython's current import *implementation* (particularly in areas that stray outside of the current documentation and PEP 302), and it isn't always clear which behaviour can be safely skipped and which needs to be faithfully replicated to avoid breaking 3rd party utilities. If the current system is setting __path__ to a string in frozen packages, I'd have to wonder how well any existing __path__ manipulation tools handle frozen packages without special-casing them. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] is the 'path' argument to an importer's find_module() just a hint?
On Mon, Oct 27, 2008 at 1:30 PM, Nick Coghlan <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] wrote: >> On 07:24 pm, [EMAIL PROTECTED] wrote: >>> On Mon, Oct 27, 2008 at 3:50 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote: Brett Cannon wrote: >> >>> Good enough for me. Then I am just going to ignore the 'path' argument >>> for frozen modules but use it to short-circuit imports for built-in >>> modules. >> >> For what it's worth, Twisted uses the __path__ attribute to facilitate >> its plugin mechanism. If this changes so that importing from frozen >> packages no longer honors __path__, then Twisted will no longer support >> plugins if the package to be plugged into is frozen. >> >> What is the motivation to change this? > > If I had to guess, I'd say Brett found some time to tinker with his > import_in_py implementation. Yep, it's called trying to prevent it from becoming true vaporware while procrastinating. =) > That has been an interesting exercise in > blowing dust out of some of the dark corners of CPython's current import > *implementation* (particularly in areas that stray outside of the > current documentation and PEP 302), and it isn't always clear which > behaviour can be safely skipped and which needs to be faithfully > replicated to avoid breaking 3rd party utilities. > You can say that again. Once I have a backwards-compatible version ready to be merged into the core I will have a PEP or two to write that will help make the semantics more uniform and easier to follow. > If the current system is setting __path__ to a string in frozen > packages, I'd have to wonder how well any existing __path__ manipulation > tools handle frozen packages without special-casing them. > I doubt anyone does. As Thomas asked, do people really even still use frozen modules? -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] is the 'path' argument to an importer's find_module() just a hint?
Brett Cannon wrote: > Beats me. At this point I am only implementing them for > backwards-compatibility, but they are exposing some inconsistencies in > how various aspects of import are implemented. If I had my way they > would be an optional thing that you would have to explicitly put on > sys.meta_path to avoid the overhead. Implementing it that way for your import_in_py project would probably be a good thing :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] is the 'path' argument to an importer's find_module() just a hint?
On 08:44 pm, [EMAIL PROTECTED] wrote: On Mon, Oct 27, 2008 at 1:30 PM, Nick Coghlan <[EMAIL PROTECTED]> wrote: If the current system is setting __path__ to a string in frozen packages, I'd have to wonder how well any existing __path__ manipulation tools handle frozen packages without special-casing them. I doubt anyone does. As Thomas asked, do people really even still use frozen modules? Well, speaking of vaporware, I know that when Divmod is discussing proprietary Python software with potential clients, Freeze often comes up. We tend to discourage its actual use, but the *possibility* of jamming all the distributed Python modules into a shared library or executable and never loading them directly from the filesystem has been an important selling point of Python in the past. I do know that a few commercial games have shipped with a big pile of frozen Python inside them; I don't know when the most recent one was. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
