[issue6727] ImportError when package is symlinked on Windows

R. David Murray added the comment: As far as I can tell from the messages, there is nothing left to do here, so I'm closing it. -- nosy: +r.david.murray resolution: -> fixed stage: -> committed/rejected status: open -> closed ___ Python tracker

[issue6727] ImportError when package is symlinked on Windows

Roundup Robot added the comment: New changeset 24369f6c4a22 by Jason R. Coombs in branch 'default': Prefer assertEqual to simply assert per recommendation in issue6727. http://hg.python.org/cpython/rev/24369f6c4a22 -- ___ Python tracker

[issue6727] ImportError when package is symlinked on Windows

Éric Araujo added the comment: (Oh BTW I put back 3.3 in versions without seeing you had removed it recently; even though the bug is not present there, the tests from 3.2 are merged into 3.3 so if I understand the use of the versions field correctly it makes sense to keep 3.3. Didn’t want yo

[issue6727] ImportError when package is symlinked on Windows

Éric Araujo added the comment: Would you mind making a small change and use assertEqual instead of an assert statement? The test suite should run the same tests under -O and -OO; there are remaining asserts in old tests but for the last year or two we’ve tried to change them. Thanks in adva

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: I've disabled the test on non-Windows systems and created issue15091 to track the issue. -- ___ Python tracker ___ ___

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: It seems that by adding the test to the default branch, the buildbots are now broken for Unix systems (they're failing the test). I'm going to await the results from the Windows buildbots, and then skip the tests on the failing platforms. -- __

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: I don't know what changed in 3.3 to address this issue; likely candidates are the VS 2010 upgrade or the rewrite of the import machinery. In either case, best I can tell, 3.3 is no longer implicated. -- ___ Python

[issue6727] ImportError when package is symlinked on Windows

Changes by Jason R. Coombs : -- versions: -Python 3.3 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://ma

[issue6727] ImportError when package is symlinked on Windows

Roundup Robot added the comment: New changeset afe67ea94bc6 by Jason R. Coombs in branch 'default': Adding test from issue6727 demonstrating that symlink import issue does not occur here in 3.3 http://hg.python.org/cpython/rev/afe67ea94bc6 -- ___ Py

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: At PyCon, I took another look at the Python 3.2 changes. Unfortunately, I found they trigger failures in other aspects of the test suite, so the approach is not viable as implemented. I plan to take another look at this at some point, but I've been utterly

[issue6727] ImportError when package is symlinked on Windows

Brian Curtin added the comment: The @support.skip_unless_symlink decorator could be helpful there. -- ___ Python tracker ___ ___ Pytho

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: The Unix tests are now passing again. Some Windows buildbots are failing because they do not have the symlink privilege. I'll add a guard in the test to not run it if it doesn't have privilege to symlink. -- ___ P

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: I see that. Thanks. I'm on it. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscr

[issue6727] ImportError when package is symlinked on Windows

Florent Xicluna added the comment: It breaks the compilation on few buildbots. Python/import.c:130:14: error: #if with no expression -- nosy: +flox ___ Python tracker ___ ___

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: I've gone ahead and pushed the changesets for Python 2.7. I'll continue to hold off on Python 3.2/3.3 until I can review. Perhaps Brian and I can review these changes at PyCon. -- ___ Python tracker

[issue6727] ImportError when package is symlinked on Windows

Changes by Jason R. Coombs : -- hgrepos: -104 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pytho

[issue6727] ImportError when package is symlinked on Windows

Roundup Robot added the comment: New changeset 6c218b9c5c4c by Jason R. Coombs in branch '2.7': Extracted Windows directory detection from NullImporter.__init__. This greatly simplifies the code and fixes issue6727. http://hg.python.org/cpython/rev/6c218b9c5c4c New changeset 92f4d4eebed1 by Ja

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: After hearing back from Microsoft support (and by proxy, the Visual Studio development team), it is clear that this issue is very low priority for them (they see it as having trivial business impact), so we cannot expect it to be fixed in the upstream librar

[issue6727] ImportError when package is symlinked on Windows

Éric Araujo added the comment: (3.1 doesn’t get non-security bug fixes either) -- versions: -Python 3.1 ___ Python tracker ___ ___ Py

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: I ported the patch to the 3.2 branch. This port required more extensive adjustments. In particular, the NullImporter no longer used (char *) for directory detection, but instead used platform-specific PyObjects. Therefore, the 3.2 diff creates a new function

[issue6727] ImportError when package is symlinked on Windows

Changes by Jason R. Coombs : Added file: http://bugs.python.org/file24251/4fdbc9f74235.diff ___ Python tracker ___ ___ Python-bugs-list mailing

[issue6727] ImportError when package is symlinked on Windows

Changes by Jason R. Coombs : -- hgrepos: +105 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: 4aaf78f0dd10.diff is the same patch as that for 2.7 but ported to 3.1. -- ___ Python tracker ___ __

[issue6727] ImportError when package is symlinked on Windows

Changes by Jason R. Coombs : Added file: http://bugs.python.org/file24250/4aaf78f0dd10.diff ___ Python tracker ___ ___ Python-bugs-list mailing

[issue6727] ImportError when package is symlinked on Windows

Changes by Jason R. Coombs : -- hgrepos: +104 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: I've created yet another patch (1cdb64480494) which adds a regression test under Python 2.7 to demonstrate that the fix works (based on test_import_symlink_package.py). -- ___ Python tracker

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: I've created yet another patch (1cdb64480494) which adds a regression test under Python 2.7 to demonstrate that the fix works (based on test_import_symlink_package.py). -- ___ Python tracker

[issue6727] ImportError when package is symlinked on Windows

Changes by Jason R. Coombs : Added file: http://bugs.python.org/file24246/1cdb64480494.diff ___ Python tracker ___ ___ Python-bugs-list mailing

[issue6727] ImportError when package is symlinked on Windows

Changes by Jason R. Coombs : Added file: http://bugs.python.org/file24245/2b7bf4e5cb9f.diff ___ Python tracker ___ ___ Python-bugs-list mailing

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: I've regenerated the diff a couple of times (once after rebasing to not undo Amaury's changes and another after correcting the whitespace). -- keywords: +needs review ___ Python tracker

[issue6727] ImportError when package is symlinked on Windows

Changes by Jason R. Coombs : Added file: http://bugs.python.org/file24231/577b717055bc.diff ___ Python tracker ___ ___ Python-bugs-list mailing

[issue6727] ImportError when package is symlinked on Windows

Changes by Jason R. Coombs : Added file: http://bugs.python.org/file24230/55d164f36389.diff ___ Python tracker ___ ___ Python-bugs-list mailing

[issue6727] ImportError when package is symlinked on Windows

Alex Regueiro added the comment: That's very good news. I suspected MS has written a wrapper over this somewhere to accommodate for this bug, and it was just a matter of using the right API. I look forward to seeing this in the next 2.7 release. -- ___

[issue6727] ImportError when package is symlinked on Windows

Amaury Forgeot d'Arc added the comment: Please regenerate your diff file, it undoes an unrelated fix :-) -- ___ Python tracker ___ ___

[issue6727] ImportError when package is symlinked on Windows

Changes by Jason R. Coombs : -- keywords: +patch Added file: http://bugs.python.org/file24229/f3c7f4243a04.diff ___ Python tracker ___

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: The MS support engineer has come back confirming the bug and suggesting using the MS C API instead of wstat for doing directory detection. It's still possible they'll fix the bug in an upcoming update, but even if they do, it will leave a window of broken sy

[issue6727] ImportError when package is symlinked on Windows

Alex Regueiro added the comment: That's great, thanks. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: I've filed a support incident with the Microsoft Support team to try to push this issue determine a fix or workaround. -- ___ Python tracker _

[issue6727] ImportError when package is symlinked on Windows

Amaury Forgeot d'Arc added the comment: > If we could get it in the next release (Python 2.8?) that would be awesome. I doubt it will (see PEP 404) but 3.3 is a good target. -- nosy: +amaury.forgeotdarc ___ Python tracker

[issue6727] ImportError when package is symlinked on Windows

Alex Regueiro added the comment: Thanks Brian; that's good to know. If we could get it in the next release (Python 2.8?) that would be awesome. -- ___ Python tracker ___ ___

[issue6727] ImportError when package is symlinked on Windows

Brian Curtin added the comment: There are a few of us, and Jason and myself have done most of the Windows symlink related work. We'll certainly get to this and have it fixed, but with no releases on the immediate horizon, there isn't a rush. This and your other symlink issue are on my radar f

[issue6727] ImportError when package is symlinked on Windows

Alex Regueiro added the comment: Well, I am making an assumption here. :-) Might have a look at a patch. Never looked at the Python runtime before. If it's all written in C, then it's possible. I'm guessing most of the Python programmers don't know much/anything about Win32 programming, so th

[issue6727] ImportError when package is symlinked on Windows

Brian Curtin added the comment: Not fixed, but if it's easy, you're welcome to fix it before we get around to it. -- ___ Python tracker ___ _

[issue6727] ImportError when package is symlinked on Windows

Alex Regueiro added the comment: Is this bug going to be fixed? I've been experiencing this for some time now, and it seems the Python team has known about it for a while too... seems like an easy fix! -- nosy: +alexreg ___ Python tracker

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: If this issue is important to you, please follow the link above to the Microsoft Connect site and vote for the issue. More votes should mean it will get more attention. Thanks. -- ___ Python tracker

[issue6727] ImportError when package is symlinked on Windows

Waldemar Kornewald added the comment: It was only with symlinked packages/folders. Symlinked files worked correctly for me. BTW, pth files won't work for our specific use-case (App Engine). -- ___ Python tracker

[issue6727] ImportError when package is symlinked on Windows

Éric Araujo added the comment: BTW, does this bug occur only with symlinked packages or also with one-file modules? -- ___ Python tracker ___ ___

[issue6727] ImportError when package is symlinked on Windows

Éric Araujo added the comment: Removing 2.6, this is not a security bug. (OT: > Could someone please fix support for symlinked packages? It's an > essential feature during development. If I correctly guess your use case, you could use a pth file during development. See the docs.) --

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: Per the suggestion in the Visual Studio forums, I posted a report against Visual Studio here: https://connect.microsoft.com/VisualStudio/feedback/details/669601/regression-calling-wstat64i32-on-symlink-directory-with-kb2467174-installed#details -- _

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: MS Connect is apparently only for projects under active development, not mature, released products. I've posted to the MSDN forums, where with my MSDN account, I can expect priority support from MS personnel. http://social.msdn.microsoft.com/Forums/en-US/vcg

[issue6727] ImportError when package is symlinked on Windows

Brian Curtin added the comment: Wow, nice analysis. http://connect.microsoft.com/ is the external Microsoft bug tracker, as far as I know. -- ___ Python tracker ___

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: Indeed, this appears to be a bug in stat64.c, specifically a regression in KB2467174. If I look at the code for _wstat64i32, it doesn't have the code that calls into _wsopen_s for symlinked files/dirs, so uses the legacy behavior to stat the target. I suspec

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: Digging deeper with the Visual Studio debugger, I discovered the following interesting outcome (run with cmd.exe): @echo off mklink /d sample sample-target mkdir sample-target echo "" > sample-target\__init__.py :: Before KB2467174, return

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: I decided to investigate further. I created another script to test the call to _wstat to try to recreate the -1 return code, but I was unable to do so. I'm attaching the script, which creates the same 'sample' package, but instead of calling import, calls _w

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: I've made some progress on this issue. Thanks to Waldemar's findings, I'm able to selectively reproduce the issue by installing/uninstalling the Visual C++ redistributable that includes KB2467174 on a Windows 7 RTM installation. So I added a patch to cpython

[issue6727] ImportError when package is symlinked on Windows

Changes by Santoso Wijaya : -- nosy: +santa4nt ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pytho

[issue6727] ImportError when package is symlinked on Windows

Waldemar Kornewald added the comment: That's really strange. We uninstalled Service Pack 1 from my friend's machine and now everything works. What I don't understand is why I don't have Service Pack 1 on my machine and why Windows Update also doesn't offer to install it. Does it have to be in

[issue6727] ImportError when package is symlinked on Windows

Waldemar Kornewald added the comment: I've tracked it down. It's caused by KB2467174 (http://support.microsoft.com/kb/2467174) and I can reliably reproduce it. Here's how to reproduce it: Install *only* the "Microsoft Visual C++ 2008 Redistributable Package (x86)" from http://www.microsoft.

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: Updated script to run under Python 3.2 as well. -- Added file: http://bugs.python.org/file21962/test_import_symlink_package.py ___ Python tracker _

[issue6727] ImportError when package is symlinked on Windows

Changes by Jason R. Coombs : Removed file: http://bugs.python.org/file21959/test_import_symlink_package.py ___ Python tracker ___ ___ Python-bu

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: Confirmed the issue exists on Python 2.7 and 3.2 -- versions: +Python 2.7, Python 3.2 ___ Python tracker ___ __

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: I updated the test script to avoid the error when the .pyc (or __pycache__) is created. -- Added file: http://bugs.python.org/file21959/test_import_symlink_package.py ___ Python tracker

[issue6727] ImportError when package is symlinked on Windows

Changes by Jason R. Coombs : Removed file: http://bugs.python.org/file14741/test_import_symlink_package.py ___ Python tracker ___ ___ Python-bu

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: Thanks for that. The output is very telling. First, it shows there's a bug in the test script where an exception occurs when it succeeds because a .pyc file is created and not properly cleaned up. Second, it demonstrates that the bug as I reported it is not

[issue6727] ImportError when package is symlinked on Windows

Waldemar Kornewald added the comment: I'm using Python 2.6.6 on Windows 7 Professional with the latest service pack. My system is pretty bare bones. Do you have Visual Studio or the Visual Studio Redistributables? Here's the output of your script (the last exception seems to be caused by the

[issue6727] ImportError when package is symlinked on Windows

Brian Curtin added the comment: Out of the patches listed, http://www.microsoft.com/technet/security/advisory/2269637.mspx is the only part that appears to be related in any way, although it doesn't specify a whole lot on the surface. The title is "Insecure Library Loading Could Allow Remote

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: And since you seem to have some systems that honor symlinked packages, can you run the attached test_import_symlink_package.py and report the results? -- ___ Python tracker _

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: That's interesting. You're saying that you've been using symlinked packages for some time and that it works for you. I filed this bug because it's never worked for me. Can you describe a little bit about the environment in which you've encountered this? Spec

[issue6727] ImportError when package is symlinked on Windows

Brian Curtin added the comment: Don't do anything you're not comfortable with. If you can get your system to whatever state it was in the past where things worked properly, feel free to dig into it. I will try to look into this situation and see if there's anything in these security updates t

[issue6727] ImportError when package is symlinked on Windows

Waldemar Kornewald added the comment: I uninstalled these three security patches: * KB2467173: http://support.microsoft.com/kb/2467173 * KB2467174: http://support.microsoft.com/kb/2467174 * KB2467175: http://support.microsoft.com/kb/2467175 Without those patches symlinked packages work for me.

[issue6727] ImportError when package is symlinked on Windows

Brian Curtin added the comment: Are you able to narrow it down to which security update(s) caused the breakage? -- ___ Python tracker ___

[issue6727] ImportError when package is symlinked on Windows

Waldemar Kornewald added the comment: Symlinked packages used to work on Windows until recently, but a few days ago Microsoft published a few security patches and things stopped working. On my local machine I uninstalled all Visual C++ Runtime library patches (they were installed automaticall

[issue6727] ImportError when package is symlinked on Windows

Changes by Brian Curtin : -- nosy: +brian.curtin ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyt

[issue6727] ImportError when package is symlinked on Windows

Changes by Andrew Svetlov : -- nosy: +asvetlov ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pytho

[issue6727] ImportError when package is symlinked on Windows

Jason R. Coombs added the comment: I agree that a proper solution might include full symlink support. However, symlink support for windows (issue1578269) is taking some time to develop and will likely never be available for Python 2. My feeling is that a symlink naive implementation (such as P

[issue6727] ImportError when package is symlinked on Windows

Nick Coghlan added the comment: Given that you had to use ctypes to create the symlinks on Windows in the first place, I'd say you're right :) Probably the first step would be to make sure the nt OS and path modules get Windows symlink support (although even that may not be enough, due to the b

[issue6727] ImportError when package is symlinked on Windows

New submission from Jason R. Coombs : Python reports an import error, even when a valid package exists on sys.path, if that package is a symlink directory and would otherwise be importable. The attached test script demonstrates the limitation on Python 2.6.2 and 3.1.0. I suspect that the underl