Re: Experimental Python interpreter snap
On 23 February 2017 at 14:45, James Henstridge wrote: > So if I installed a package to $SNAP_USER_DATA for my > "python36-jamesh.python3" interpreter, the files would end up > somewhere under ~/snap/python36-jamesh/. > > If we then look at my simple hello-world example snap that uses the > content interface to access the interpreter, $SNAP_USER_DATA now > points to a location under ~/snap/hello-world/. So it wouldn't see > the additional packages installed for "python36-jamesh.python3". In > fact, the hello-world snap doesn't even have permission to read files > under ~/snap/python36-jamesh, even if I put that directory on > sys.path. Yes, exactly. If I use the python36-jamesh.python3 interpreter from my main shell, I can pip install libraries into $SNAP_USER_DATA and use them. They will not be seen or cause conflicts if I use the /usr/bin/python3, which is what I'm suggesting is an improvement. And they will not be seen by or cause conflicts with snaps using the content interface, no matter if they are confined or classic, which is also a good thing because they will be embedding their dependencies. -- Stuart Bishop -- Snapcraft mailing list Snapcraft@lists.snapcraft.io Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/snapcraft
Re: Position-independent code and Ubuntu 16.10
On 22/02/17 00:39, Joseph Rushton Wakeling wrote: OK, fair enough. In my case I think it was the C-library parts of the D standard library that were being compiled without PIC. Seems OK to assume this may have been a project-specific thing, then (which is now fixed upstream). Just for the record: this issue is fixed in the most recent revision of the package. Seth, thanks for your help and advice! -- Snapcraft mailing list Snapcraft@lists.snapcraft.io Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/snapcraft
Further ABI-related issues for ldc2 snap on 14.04 ... ?
Hello all, I've encountered a couple of further issues with the ldc2 compiler snap when it's installed on a 14.04 system. I'm sharing here to confirm if these are indeed ABI issues which might be fixable in terms of how snapcraft constructs the package. The first [1] relates to zlib, which is included as part of the D standard library. This is built from a copy of the zlib C source and linked in to the rest of the library. However, if I try to build a program that makes use of the zlib library modules, linking fails with the following error: /usr/bin/gcc zlibtest.o -o zlibtest -L/snap/ldc2/x1/bin/../lib -lphobos2-ldc -ldruntime-ldc -Wl,--gc-sections -lrt -ldl -lpthread -lm -m64 /usr/bin/ld: /snap/ldc2/x1/bin/../lib/libphobos2-ldc.a(zutil.c.o): unrecognized relocation (0x2a) in section `.text' /usr/bin/ld: final link failed: Bad value collect2: error: ld returned 1 exit status Error: /usr/bin/gcc failed with status: 1 This matches the error message observed in a past Ubuntu issue related to binutils: https://bugs.launchpad.net/ubuntu/+source/binutils/+bug/1545653 which would suggest that there's a problematic mismatch between the binutils used to build snap packages versus the one available on 14.04. The second issue [2] relates to a plugin for the gold linker that is built for LDC to enable it to use link-time optimization. This works fine on 16.04, 16.10 and 17.04, but requesting LTO on 14.04 results in linking failing: /usr/bin/gcc hello.o -o hello -Xlinker -plugin -Xlinker /snap/ldc2/x1/lib/LLVMgold-ldc.so -Xlinker -plugin-opt=O0 -L/snap/ldc2/x1/bin/../lib -lphobos2-ldc -ldruntime-ldc -Wl,--gc-sections -lrt -ldl -lpthread -lm -m64 /usr/bin/ld: /snap/ldc2/x1/lib/LLVMgold-ldc.so: error loading plugin: /snap/core/current/lib/x86_64-linux-gnu/libpthread.so.0: symbol __libc_vfork, version GLIBC_PRIVATE not defined in file libc.so.6 with link time reference collect2: error: ld returned 1 exit status Error: /usr/bin/gcc failed with status: 1 ... which looks to me like it is probably a mismatch between the libc used to build the plugin, versus the libc of either the host system or the core snap? I would presume that both the above would need a fix either snapcraft-side or core snap side ... ? Can anyone advise/assist? Thanks & best wishes, -- Joe [1] https://github.com/ldc-developers/ldc2.snap/issues/17 [2] https://github.com/ldc-developers/ldc2.snap/issues/18 -- Snapcraft mailing list Snapcraft@lists.snapcraft.io Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/snapcraft
systemd-resolved and snaps
Hey all. I've received a bug report on a snap where the user was running a 16.10 Server install with the snap in question, and getting DNS errors. I've distilled the problem as much as I can but I cannot for the life of me figure out what's happening, so I thought maybe the list could point me in the right direction. Prerequisites = I have a demo snap (a standalone snapcraft.yaml) that will demonstrate this issue[1]. Ubuntu 16.10 Server uses systemd-resolved, which means its /etc/resolv.conf contains a single nameserver: nameserver 127.0.0.53 If you have others there, comment them out for the time being. Steps to reproduce == 1. Build and install the `resolved-test` snap[1]. It exposes two apps, `test` (which is a python2 script uses the requests lib) and `host` which is just the `host` utility from bind9-host. 2. With 127.0.0.53 as the only nameserver, run `resolved-test.test`. Note that it fails with "Name or service not known." 3. With 127.0.0.53 as the only nameserver, run `resolved-test.host acme-staging.api.letsencrypt.org`. Note that it works perfectly fine. At this point, one might suspect that this is an issue in the requests lib. So: 4. Run the test from outside the snap: `PYTHONPATH=/snap/resolved-test/current/lib/python2.7/site-packages /snap/resolved-test/current/usr/bin/python /snap/resolved-test/current/bin/test.py`. Note that this works fine (you should see a dict printed). 5. Now add another nameserver to /etc/resolv.conf (I used `nameserver 8.8.8.8`), and run `resolved-test.test`. Note that it works fine. Some analysis = Note that when I run step (1) I get [2] in the syslog. When I run step (2) I get [3] in the syslog. When I run step (4) I get nothing in the syslog. I'm at a loss here. Installing with devmode doesn't change these results. `host` always seems to work fine, but `test` never works with the systemd-resolved IP address. Has anyone else encountered this? Any advice would be appreciated. [1]: http://pastebin.ubuntu.com/24055388/ [2]: http://pastebin.ubuntu.com/24055470/ [3]: http://pastebin.ubuntu.com/24055466/ -- Kyle Fazzari (kyrofa) Software Engineer Canonical Ltd. k...@canonical.com signature.asc Description: OpenPGP digital signature -- Snapcraft mailing list Snapcraft@lists.snapcraft.io Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/snapcraft
Re: systemd-resolved and snaps
On Thu, Feb 23, 2017 at 01:58:07PM -0800, Kyle Fazzari wrote: > Hey all. > > I've received a bug report on a snap where the user was running a 16.10 > Server install with the snap in question, and getting DNS errors. I've > distilled the problem as much as I can but I cannot for the life of me > figure out what's happening, so I thought maybe the list could point me > in the right direction. > > > Prerequisites > = > > I have a demo snap (a standalone snapcraft.yaml) that will demonstrate > this issue[1]. > > Ubuntu 16.10 Server uses systemd-resolved, which means its > /etc/resolv.conf contains a single nameserver: > > nameserver 127.0.0.53 > > If you have others there, comment them out for the time being. > > > Steps to reproduce > == > > 1. Build and install the `resolved-test` snap[1]. It exposes two apps, > `test` (which is a python2 script uses the requests lib) and `host` > which is just the `host` utility from bind9-host. > > 2. With 127.0.0.53 as the only nameserver, run `resolved-test.test`. > Note that it fails with "Name or service not known." > > 3. With 127.0.0.53 as the only nameserver, run `resolved-test.host > acme-staging.api.letsencrypt.org`. Note that it works perfectly fine. > > At this point, one might suspect that this is an issue in the requests > lib. So: > > 4. Run the test from outside the snap: > `PYTHONPATH=/snap/resolved-test/current/lib/python2.7/site-packages > /snap/resolved-test/current/usr/bin/python > /snap/resolved-test/current/bin/test.py`. Note that this works fine (you > should see a dict printed). > > 5. Now add another nameserver to /etc/resolv.conf (I used `nameserver > 8.8.8.8`), and run `resolved-test.test`. Note that it works fine. > > > Some analysis > = > > Note that when I run step (1) I get [2] in the syslog. When I run step > (2) I get [3] in the syslog. When I run step (4) I get nothing in the > syslog. > > I'm at a loss here. Installing with devmode doesn't change these > results. `host` always seems to work fine, but `test` never works with > the systemd-resolved IP address. Has anyone else encountered this? Any > advice would be appreciated. > > [1]: http://pastebin.ubuntu.com/24055388/ > [2]: http://pastebin.ubuntu.com/24055470/ > [3]: http://pastebin.ubuntu.com/24055466/ Hey there, Since acme-staging.api.letsencrypt.org is a CNAME, that may be related to: - https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1647031 - https://github.com/systemd/systemd/issues/3826 What happens if you try with "host letsencrypt.org" and "https://letsencrypt.org"; (that's an A++MX instead of CNAME)? -- Stéphane Graber Ubuntu developer http://www.ubuntu.com signature.asc Description: PGP signature -- Snapcraft mailing list Snapcraft@lists.snapcraft.io Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/snapcraft
Re: systemd-resolved and snaps
Hi Kyle, On Thu, Feb 23, 2017 at 01:58:07PM -0800, Kyle Fazzari wrote: > Hey all. > I've received a bug report on a snap where the user was running a 16.10 > Server install with the snap in question, and getting DNS errors. I've > distilled the problem as much as I can but I cannot for the life of me > figure out what's happening, so I thought maybe the list could point me > in the right direction. > Prerequisites > = > > I have a demo snap (a standalone snapcraft.yaml) that will demonstrate > this issue[1]. > > Ubuntu 16.10 Server uses systemd-resolved, which means its > /etc/resolv.conf contains a single nameserver: > > nameserver 127.0.0.53 > > If you have others there, comment them out for the time being. > > > Steps to reproduce > == > > 1. Build and install the `resolved-test` snap[1]. It exposes two apps, > `test` (which is a python2 script uses the requests lib) and `host` > which is just the `host` utility from bind9-host. > > 2. With 127.0.0.53 as the only nameserver, run `resolved-test.test`. > Note that it fails with "Name or service not known." acme-staging.api.letsencrypt.org is a CNAME. You are hitting bug #1647031, which we encountered when trying to roll out systemd-resolved by default for 17.04. This took a while to work through, but the fix has finally landed in zesty as of a week ago; we should now SRU the upstream change back to yakkety. (We should also SRU it back to xenial, but xenial needs a more complete backport of fixes to resolved, not just a cherry-pick of this one fix.) Dimitri, could you handle this backport to yakkety? Since unlike the Desktop, Ubuntu Server does not use dnsmasq by default (which would override resolved), this is a rather important bug there. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature -- Snapcraft mailing list Snapcraft@lists.snapcraft.io Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/snapcraft
Re: systemd-resolved and snaps
On 02/23/2017 02:26 PM, Steve Langasek wrote: > Hi Kyle, > > On Thu, Feb 23, 2017 at 01:58:07PM -0800, Kyle Fazzari wrote: >> Hey all. > >> I've received a bug report on a snap where the user was running a 16.10 >> Server install with the snap in question, and getting DNS errors. I've >> distilled the problem as much as I can but I cannot for the life of me >> figure out what's happening, so I thought maybe the list could point me >> in the right direction. > >> Prerequisites >> = >> >> I have a demo snap (a standalone snapcraft.yaml) that will demonstrate >> this issue[1]. >> >> Ubuntu 16.10 Server uses systemd-resolved, which means its >> /etc/resolv.conf contains a single nameserver: >> >> nameserver 127.0.0.53 >> >> If you have others there, comment them out for the time being. >> >> >> Steps to reproduce >> == >> >> 1. Build and install the `resolved-test` snap[1]. It exposes two apps, >> `test` (which is a python2 script uses the requests lib) and `host` >> which is just the `host` utility from bind9-host. >> >> 2. With 127.0.0.53 as the only nameserver, run `resolved-test.test`. >> Note that it fails with "Name or service not known." > > acme-staging.api.letsencrypt.org is a CNAME. You are hitting bug #1647031, > which we encountered when trying to roll out systemd-resolved by default for > 17.04. This took a while to work through, but the fix has finally landed in > zesty as of a week ago; we should now SRU the upstream change back to > yakkety. (We should also SRU it back to xenial, but xenial needs a more > complete backport of fixes to resolved, not just a cherry-pick of this one > fix.) > > Dimitri, could you handle this backport to yakkety? Since unlike the > Desktop, Ubuntu Server does not use dnsmasq by default (which would override > resolved), this is a rather important bug there. Stephane, Steve, thanks for the quick responses. That explanation makes sense until I consider step (4): How did that succeed? I did note that it didn't actually seem to hit systemd-resolved (due to the lack of a syslog entry), but I'm not sure why. -- Kyle Fazzari (kyrofa) Software Engineer Canonical Ltd. k...@canonical.com signature.asc Description: OpenPGP digital signature -- Snapcraft mailing list Snapcraft@lists.snapcraft.io Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/snapcraft
Creating and running tests for a snapcraft plugin
Hello all, I have started working on a snapcraft plugin for dub, a build/package manager for the D programming language. The plugin itself is ready (at least in a working draft), and now I'm working on the tests. Is there a simple way to run the tests for a single plugin, rather than the whole body of tests? The snapcraft HACKING.md doesn't offer any advice on this point. I'm not much of a pythonist, so apologies if this is a daft question and the answer is as simple as `python3 snapcraft/tests/test_dub.py` (but currently at least, even trying that fails with an ImportError, so I assume there's something else I have to do). Thanks & best wishes, -- Joe -- Snapcraft mailing list Snapcraft@lists.snapcraft.io Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/snapcraft
Re: systemd-resolved and snaps
On Thu, Feb 23, 2017 at 02:39:27PM -0800, Kyle Fazzari wrote: > On 02/23/2017 02:26 PM, Steve Langasek wrote: > > acme-staging.api.letsencrypt.org is a CNAME. You are hitting bug #1647031, > > which we encountered when trying to roll out systemd-resolved by default for > > 17.04. This took a while to work through, but the fix has finally landed in > > zesty as of a week ago; we should now SRU the upstream change back to > > yakkety. (We should also SRU it back to xenial, but xenial needs a more > > complete backport of fixes to resolved, not just a cherry-pick of this one > > fix.) > > > > Dimitri, could you handle this backport to yakkety? Since unlike the > > Desktop, Ubuntu Server does not use dnsmasq by default (which would override > > resolved), this is a rather important bug there. > > Stephane, Steve, thanks for the quick responses. That explanation makes > sense until I consider step (4): How did that succeed? I did note that > it didn't actually seem to hit systemd-resolved (due to the lack of a > syslog entry), but I'm not sure why. For context, 4 was: 4. Run the test from outside the snap: `PYTHONPATH=/snap/resolved-test/current/lib/python2.7/site-packages /snap/resolved-test/current/usr/bin/python /snap/resolved-test/current/bin/test.py`. Note that this works fine (you should see a dict printed). At a guess, I would say that outside the snap you have the libnss-resolve package installed and active, such that python resolves it via the resolved native dbus interface instead of via the DNS stub resolver on port 53. These behavior differences are a reason that we should NOT use the libnss-resolve package, but instead sort out any bugs in the stub resolver and use it consistently everywhere. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature -- Snapcraft mailing list Snapcraft@lists.snapcraft.io Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/snapcraft
Heads up: change in snapd test workers
Hello there, We're starting to blow up the 50 minutes time quota provided by Travis during CI of master, which means PRs may now fail for no reason other than taking too long. On quick inspection, prepare.sh claims to have fixed the original problem that motivated the ubuntu-core-16-64-fixme spread system, and looking at test results using two workers on 14.04 and ubuntu-core-64 might cut about 20 minutes of testing time. I've pushed a PR and validated the theory in practice: https://github.com/snapcore/snapd/pull/2928 With this small tweak, we've indeed cut down on about 20 minutes of testing time, going back to ~30 minutes. The main cost of this change is just one more machine used per full test run, and the old machine that was specific to the one bogus test we had is now being used as a second standard worker that can parallelize all tests targeting ubuntu-core-16-64. We're keeping that one machine allocated for more time, but that pays off because first worker that used to take 50+ minutes now takes 30 as well. Seems to work, but I'm leaving on holiday, so two points to keep in mind: - The bug that seemed fixed is perhaps not fixed, and I was just lucky. If that is the case, have a look at the PR above and either revert the change or fix the remaining bits. - Instant contention increases. We're now running at 8 machines per full run (16.04-64+16.04-32+14.04+core-16, times two). On the down side, we may see more of the "no machines available" complaint. That said, the whole run takes less time, so trailing machines get deallocated faster. I expect the net effect here to be positive because Travis also has a maximum number of jobs, so even if we have machines available, that job taking 50 minutes may be holding back the follow ups. Anyway, just wanted to share these details as this was a last minute change merged with almost no discussion. If you see issues, that's where to look. Have fun! gustavo @ http://niemeyer.net -- Snapcraft mailing list Snapcraft@lists.snapcraft.io Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/snapcraft
