Python 3.6 fails to install to non-standard directory under Linux

2018-03-02 Thread bsferrazza
I posted this to Stackoverflow and the original post can be seen here. I'll try 
and copy and past the contents below. Thank you for your help!

https://stackoverflow.com/questions/49074327/python-3-6-fails-to-install-to-non-standard-directory-under-linux

I have a completely insulated boostrapped toolchain+binaries setup (located 
under /home/tools) that has been able to build everything that I throw at it. 
That includes Python 2.7.14, which built without issue with a simple --prefix. 
The lone exception is Python 3.6.4. I had to edit the configure and setup.py 
script to patch some of the hardcoded paths that were set to standard /usr 
directories (the link to ncursesw was notable). It completely compiled and 
built correctly, except failed during the 'make install' installation of pip. 
The installation scripts create a pip-build-* directory under /tmp and the 
source of the failure seems to be centered around that. It appears when the 
directory is first created, it's made with the proper permission mask. But 
after failure, I can see it's only user writable (not even readable or 
executable).
***

d-w--- 4 myuser   eng  4096 Mar  1 00:56 pip-build-xh7onsny/

***
Here's the section of the strace where it first creates the pip-build-* 
directory.
***

[pid 19771] lstat("/home/myuser/.cache/pip", {st_mode=S_IFDIR|0700, 
st_size=4096, ...}) = 0
[pid 19771] geteuid()   = 10345
[pid 19771] access("/home/myuser/.cache/pip", W_OK) = 0
[pid 19771] mkdir("/tmp/pip-build-xh7onsny", 0700) = 0
[pid 19771] lstat("/tmp", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=28672, 
...}) = 0
[pid 19771] lstat("/tmp/pip-build-xh7onsny", {st_mode=S_IFDIR|0700, 
st_size=4096, ...}) = 0
[pid 19771] getcwd("/nfs/home/myuser/lfs/sources/Python-3.6.4"..., 1024) = 
46


Here's the end of the strace that shows where it Error'ed out.


[pid 19771] munmap(0x2ac5506c, 4096) = 0
[pid 19771] stat("/nfs/home/myuser/lfs/sources/Python-3.6.4/Lib/shutil.py", 
{st_mode=S_IFREG|0644, st_size=40227, ...}) = 0
[pid 19771] open("/nfs/home/myuser/lfs/sources/Python-3.6.4/Lib/shutil.py", 
O_RDONLY|0x8 /* O_??? */) = 4
[pid 19771] ioctl(4, FIOCLEX)   = 0
[pid 19771] fstat(4, {st_mode=S_IFREG|0644, st_size=40227, ...}) = 0
[pid 19771] ioctl(4, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff38a58460) = -1 ENOTTY 
(Inappropriate ioctl for device)
[pid 19771] lseek(4, 0, SEEK_CUR)   = 0
[pid 19771] read(4, "\"\"\"Utility functions for copying"..., 32768) = 32768
[pid 19771] lseek(4, 0, SEEK_CUR)   = 32768
[pid 19771] read(4, "e, extract_dir=None, format=None"..., 8192) = 7459
[pid 19771] read(4, "", 8192)   = 0
[pid 19771] close(4)= 0
[pid 19771] ioctl(2, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff38a59f40) = -1 EINVAL 
(Invalid argument)
[pid 19771] write(2, "Exception:\nTraceback (most recen"..., 1739Exception:
Traceback (most recent call last):
  File "/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/basecommand.py", 
line 215, in main
status = self.run(options, args)
  File 
"/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/commands/install.py", line 
385, in run
requirement_set.cleanup_files()
  File "/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/utils/build.py", 
line 38, in __exit__
self.cleanup()
  File "/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/utils/build.py", 
line 42, in cleanup
rmtree(self.name)
  File 
"/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/retrying.py", line 
49, in wrapped_f
return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File 
"/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/retrying.py", line 
212, in call
raise attempt.get()
  File 
"/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/retrying.py", line 
247, in get
six.reraise(self.value[0], self.value[1], self.value[2])
  File "/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/six.py", 
line 686, in reraise
raise value
  File 
"/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/retrying.py", line 
200, in call
attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/utils/__init__.py", 
line 102, in rmtree
onerror=rmtree_errorhandler)
  File "/nfs/home/myuser/lfs/sources/Python-3.6.4/Lib/shutil.py", line 476, in 
rmtree
onerror(os.lstat, path, sys.exc_info())
  File "/nfs/home/myuser/lfs/sources/Python-3.6.4/Lib/shutil.py", line 474, in 
rmtree
fd = os.open(path, os.O_RDONLY)
PermissionError: [Errno 13] Permission denied: '/tmp/pip-build-xh7onsny'
) = 1739
[pid 19771] lstat("/tmp/tmpojuc47gy", {st_mode=S_IFDIR|0700, st_size=4096, 
...}) = 0
[pid 19771] open("/tmp/tmpojuc47gy", O_RDONLY|0x8 /* O_??? */) = 4
[pid 19771] ioctl(4, FIOCLEX)   = 0
[pid 19771] fstat(4, {st_mode=S_IFDIR|0700, st_size=4096, ...}) =

Re: Python 3.6 fails to install to non-standard directory under Linux

2018-03-02 Thread bsferrazza
Here's my configure I need to set ac_cv_fun_utimensat=no and 
ac_cv_func_futimens=no because presumably the file-system or kernel on my 
system doesn't support nanosecond timestamps. With these options, and patching 
the configure/setup.py files to remove references to /usr/lib/ncursesw and 
replace with my /home/tools/lib path, everything builds just fine. It's during 
the 'make install' phase of pip that it fails.

./configure --prefix=/home/tools \
  --enable-shared \
  --with-system-expat \
  --with-system-ffi \
  --with-ensurepip=install \
  ac_cv_func_utimensat=no \
  ac_cv_func_futimens=no
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python 3.6 fails to install to non-standard directory under Linux (Posting On Python-List Prohibited)

2018-03-02 Thread bsferrazza
On Friday, March 2, 2018 at 12:27:17 PM UTC-8, Lawrence D’Oliveiro wrote:
> On Saturday, March 3, 2018 at 6:42:05 AM UTC+13, bsfer...@avnera.com wrote:
> 
> > ...
> >   File "/nfs/home/myuser/lfs/sources/Python-3.6.4/Lib/shutil.py", line 476, 
> > in rmtree
> > ...
> 
> NFS trouble?
> 
> I have had builds give trouble when done on directories mounted via NFS, yet 
> work perfectly on local directories.


Hmm. That could very well be the case. I know the /home directory is NFS 
mounted, though not sure about /tmp.

I'm starting to get the impression that everything is actually OK, and that 
these errors are only on the cleanup after installation. Even with the meson 
install. This bugs.python.org/issue23346 bug got me in the right path. When I 
comment out the _use_fd_functions line that ends up setting it to True and 
replace it with a direct assignment to False, everything works ok. The rmtree 
function instead returns _rmtree_unsafe(path, onerror) when _use_fd_functions 
is false and that eliminates the error I was seeing.
-- 
https://mail.python.org/mailman/listinfo/python-list