Re: [yocto] [meta-security][PATCH] keynote: update the SRC_URI

2017-09-30 Thread Huang, Jie (Jackie)


> -Original Message-
> From: yocto-boun...@yoctoproject.org [mailto:yocto-
> boun...@yoctoproject.org] On Behalf Of Randy MacLeod
> Sent: Saturday, September 30, 2017 11:10
> To: Du, Dengke; yocto@yoctoproject.org
> Subject: Re: [yocto] [meta-security][PATCH] keynote: update the SRC_URI
> 
> On 2017-09-29 09:55 PM, Dengke Du wrote:
> > The old URL can't be available, give the new URL to keynote.
> > The project already moved to:
> >
> >  https://sourceforge.net/projects/keynote-2-3/
> >
> > Signed-off-by: Dengke Du 
> > ---
> >   recipes-security/keynote/keynote_2.3.bb | 9 ++---
> >   1 file changed, 6 insertions(+), 3 deletions(-)
> >
> > diff --git a/recipes-security/keynote/keynote_2.3.bb b/recipes-
> security/keynote/keynote_2.3.bb
> > index b1df880..23e75e4 100644
> > --- a/recipes-security/keynote/keynote_2.3.bb
> > +++ b/recipes-security/keynote/keynote_2.3.bb
> > @@ -9,16 +9,19 @@ SECTION = "security"
> >   LICENSE = "ISC"
> >   LIC_FILES_CHKSUM =
> "file://LICENSE;md5=3a265095c549c1808686a676f2699c98"
> >
> > -SRC_URI = "http://www.cs.columbia.edu/~angelos/Code/${BPN}.tar.gz \
> > +MAIN_ID = "${@d.getVar('PV').split('.')[0]}"
> > +MINOR_ID = "${@d.getVar('PV').split('.')[1]}"
> > +SRC_URI = "${SOURCEFORGE_MIRROR}/project/${PN}-${MAIN_ID}-
> ${MINOR_ID}/${PN}_${PV}.tar.gz \

s/PN/BPN/

> >  file://configure-remove-hardcode-path.patch \
> >  file://makefile-add-ldflags.patch \
> >  file://run-ptest \
> >   "
> > +S = "${WORKDIR}/${PN}-${PV}+dfsg.orig"

s/PN/BPN/

Dengke, please fix and re-send.

Thanks,
Jackie

> >
> >   inherit autotools-brokensep ptest
> >
> > -SRC_URI[md5sum] = "ba58a0297c421dc6aa671e6b753ef695"
> > -SRC_URI[sha256sum] =
> "62f7a9d57ceb6bcdd47b604b637a7ac8ed337cef0ab02f1fa28b7e61c9b15821"
> > +SRC_URI[md5sum] = "a14553e6ad921b5c85026ce5bec3afe7"
> > +SRC_URI[sha256sum] =
> "38d2acfa1c3630a07adcb5c8fe92d2aef7f0e6d242b8998b2bbb1c6e4c408d46"
> 
> Denke tells me that the source is identical but docs were added
> so the checksums have changed.
> 
> ../Randy
> 
> >
> >   DEPENDS = "flex openssl"
> >
> >
> 
> 
> --
> # Randy MacLeod.  WR Linux
> # Wind River an Intel Company
> --
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto


[yocto] INTEL E3900 Apollo Lake I (APL-I) with U-Boot and YOCTO?

2017-09-30 Thread Zoran Stojsavljevic
Hello to the YOCTO Community,

I am curious about the following HW/FW configuration, for the YOCTO Project?

Did anybody managed to run the following: INTEL Atom E3900 APL-I with
U-Boot and YOCTO on the top of it?

E3900 -> IFWI -> MBR -> U-Boot -> YOCTO ?!

Here are some explanations regarding the terms:

*IFWI is the Intel FirmWare Interface, a binary blob loaded from the eMMC
boot partition that executes a secondary loader (in this case U-Boot) from
the main eMMC. IFWI blobs for the APL-I are provided by Intel and are
specific for different flavors of the MID silicon.*
*Normal IFWI eMMC boot process*

   1. *On-chip boot rom inits eMMC and loads IFWI from the MMC boot
   partitions*
   2. *IFWI looks for OSIP header at top of eMMC (MBR boot block)*
   3. *The header directs IFWI to the start, size, load address, and entry
   of U-Boot in eMMC*
   4. *(need clarification) If u-boot is not found, try the alt u-boot
   image at 5MB into the eMMC*
   5. *U-Boot is loaded into RAM and executed*

*OSIP stands for OS Image Profile, and it is nothing more and less than
INTEL name for very known old fashion MBR, considering DATA structure.*

Thank you in advance,
Zoran
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto


[yocto] RFC: autotooler: generation of "configure.ac" and "Makefile.am" using Kconfig

2017-09-30 Thread Ulf Samuelsson

Have a need to convert a large number of libraries to use autotools.

I came up with the following idea:

I create an Kconfig based configuration system, where you define a 
number of configuration items needed by configure.ac and Makefile.am


You choose if you want to build an application, a shared library or a 
static library.


The build normally depends on other libraries, and you checkmarks
which library you are interested in from a (small) number of supported 
libraries.

You can likewise select to support options like "--enable-openssl",
again from a (small) number of supported options.

If you have stuff not supported, the autotooler will include Config.in
files allowing you to add your own items.

Once the configuration is complete, you will compile an application
which will generate "configure.ac" from the configuration file.

Again, if you have stuff not supported in the code, you can add it
in a user include file.

The generator will read a user-headers.inc file and
generate an AC_CHECK_HEADERS entry.


The autotooler is available on
https://www.github.com/emagii/autotooler.git

===
Currently, it does not generate any Makefile.am files, but
the idea is to separate the source files into
* C source
* Public Headers
* Local Headers

A tool would then scan the directories and classify them accordingly,
so Makefile.am files should be possible to generate fairly easy.

===
A final step would clone a empty git tree, and populate it
with all the files needed for a complete package.
===

Comments appreciated.

Best Regards
Ulf Samuelsson
===
Here is a typical "autoconh.h" generated by Kconfig.

/*
 * Automatically generated C config: don't edit
 * Sat Sep 30 16:27:24 2017
 */
#define CONFIG_SHARED_LIB 1
#define CONFIG_PTHREAD 1
#define CONFIG_LIBRARY 1
#define CONFIG_PROJECT "myproject"
#define CONFIG_WORKDIR "${HOME}/projects/autotooler"
#define CONFIG_OPENSSL 1
#define CONFIG_CURL 1
#define CONFIG_COPYRIGHT_DATE "2017"
#define CONFIG_HOMEPAGE "http://www.emagii.com/";
#define CONFIG_EXAMPLES 1
#define CONFIG_DEBUG 1
#define HAVE_DOT_CONFIG 1
#define CONFIG_AC_PRERQ "2.59"
#define CONFIG_BOOST 1
#define CONFIG_OS_LINUX 1
#define CONFIG_LIBRARY_NAME "libyocto"
#define CONFIG_AUTHOR_EMAIL "u...@emagii.com"
#define CONFIG_LIBRARY_VERSION "1.0"
#define CONFIG_AC_CONFIG_HEADER "src/include/config.h"
#define CONFIG_OS "Linux"
#define CONFIG_EXAMPLES_VAR "examples"
#define CONFIG_AC_CONFIG_MACRO_DIR "m4"
#define CONFIG_DEBUG_VAR "debug"
#define CONFIG_PROJECT_TYPE "library"
#define CONFIG_SRCDIR "src"
#define CONFIG_SRC_URI_REPO ""
#define CONFIG_AUTHOR "Ulf Samuelsson"
#define CONFIG_SRC_URI_HOST "https://www.github.com/emagii";
#define CONFIG_CURLPP 1
#define CONFIG_AM_INIT_AUTOMAKE "1.10 -Wall no-define"

The application will from this information generate:

AC_INIT([libyocto],
[1.0],
[Ulf Samuelsson],
[libyocto-1.0],
[http://www.emagii.com/])

AC_PRERQ([2.59])

AC_CONFIG_HEADER([src/include/config.h])

AC_CONFIG_SRCDIR([src])

AC_PROG_CPP
AC_PROG_CC
AC_INIT_AUTOMAKE([1.10 -Wall no-define])

AC_PROG_MAKE_SET
AM_MAINTAINER_MODE
AC_HEADER_STDC
AC_ENABLE_SHARED
AC_DISABLE_STATIC
AC_PROG_LIBTOOL
AC_HEADER_STDC

AC_CHECK_HEADERS( \
ctype.h \
errno.h \
fcntl.h \
openssl/bio.h \
openssl/err.h \
openssl/opensslconf.h \
openssl/rand.h \
openssl/ssl.h \
pthread.h \
stdio.h \
stdlib.h \
stdarg.h \
stdint.h \
string.h \
signal.h \
sys/ioctl.h \
sys/socket.h \
sys/time.h \
sys/types.h \
time.h \
unistd.h \
 \
,
dnl to do if not found
[],
dnl to do if not found
[],
dnl default includes
[
#ifdef HAVE_SYS_TYPES_H
#include 
#endif
#ifdef HAVE_SYS_TIME_H
#include 
#endif
dnl We do this default-include simply to make sure that the nameser_compat.h
dnl header *REALLY* can be include after the new nameser.h. It seems AIX 5.1
dnl (and others?) is not designed to allow this.
#ifdef HAVE_ARPA_NAMESER_H
#include 
#endif

dnl *Sigh* these are needed in order for net/if.h to get properly detected.
#ifdef HAVE_SYS_SOCKET_H
#include 
#endif
#ifdef HAVE_NETINET_IN_H
#include 
#endif
]
)

#  Boost Libraries
AC_ARG_WITH([boost-include-path],
	[AS_HELP_STRING([--with-boost-include-path],[location of the Boost 
headers, defaults to /usr/include/boost])],

[CXXFLAGS_BOOST="-I$withval"],
[CXXFLAGS_BOOST="-I/usr/include/boost"])
AC_SUBST([BOOST_CFLAGS])

AC_ARG_WITH([boost-lib-path],
	[AS_HELP_STRING([--with-boost-lib-path],[location of the Boost 
libraries, defaults to /usr/lib])],
	[BOOST_LIBS="-L$withval" -lboost_system -lb

Re: [yocto] poky using native pkg_resources module

2017-09-30 Thread Ulf Samuelsson



Den 2017-09-29 kl. 20:18, skrev Alejandro Hernandez:

Hey Ulf,



On 09/29/2017 04:05 AM, Ulf Samuelsson wrote:

It looks like the pkg_resource module is using the native installation


I cloned poky early september, and could complete a build.

When I clone poky today, the build fails.

As part of "do_image_wic" from image_types_wic.bbclass

A python program in 'bmap-tools-native' is executing

"from pkg_resources import load_entry_point"

grep'ing for pkg_resources in bmap-tools-native reveals:

bmaptool:    from pkg_resources import load_entry_point
easy3_install:    from pkg_resources import load_entry_point
easy_install-3.5:    from pkg_resources import load_entry_point
=
pkg_resources.py is not available inside poky at the moment.

It is (or used to be) part of python-setuptools.


Correct, it was and still is part of setuptools


On Ubuntu, it is present in: 
"http://archive.ubuntu.com/ubuntu/pool/main/p/python-setuptools/python-setuptools_3.3.orig.tar.gz"; 


and used in Ubuntu 14.04 and 16.04 to generate the
"python3-pkg-resource" package.

When python-setuptools is generated by Yocto, it is partly
a class, but the python-setuptools-native recipe downloads
"https://files.pythonhosted.org/packages/source/s/setuptools/setuptools-26.2.7.zip"; 



This should be setuptools-36.2.7 btw


which does not contain "pkg_resources.py"


Its not partly a class, it uses a class to install the package along 
with distutils.


=
When bmaptool imports pkg_resources, it will not find anything in the
yocto build, but will find the native 
"/usr/lib/python3/dist-packages/pkg_resources.py" from


This requires bmap-tools == 3.4, but the native bmap-tools in 
Ubuntu-14.04 is 2.5, and in Ubuntu-16.04 it is 3.2 so the build fails.


Maybe Ubuntu-17.04 supports bmap-tools 3.4 and it will build there,
but using the native pkg_resource is of course not a good idea?


There is something wrong with your build although I am not sure what it 
is, pkg_resources is still part of python(3)-setuptools, in fact you can 
check that in several ways


if you do a devshell on python3-setuptools-native you can see the 
extracted package contains the pkg_resources folder (so it is still part 
of the package)

>
I do: "bitbake  -c devshell python3-setuptools-native"
A terminal windows opens in 
"tmp/work/x86_64-linux/python3-setuptools-native/36.2.7-r0/setuptools-36.2.7"


ls
bootstrap.py  easy_install.py  PKG-INFO   setuptools
build launcher.c   pkg_resources  setuptools.egg-info
CHANGES.rst   LICENSE  pytest.ini tests
conftest.py   MANIFEST.in  README.rst tox.ini
dist  msvc-build-launcher.cmd  setup.cfg
docs  pavement.py  setup.py

so the "pkg_resources" folder is present.

cd pkg_resources ; ls -l
-rw-r--r-- 1 ulf ulf  12211 Aug  2 00:40 api_tests.txt
drwxrwxr-x 3 ulf ulf   4096 Sep 30 17:35 extern
-rw-r--r-- 1 ulf ulf 104650 Aug  2 00:40 __init__.py
-rw-r--r-- 1 ulf ulf600 Aug  2 00:40 py31compat.py
drwxrwxr-x 2 ulf ulf   4096 Sep 30 17:35 __pycache__
drwxrwxr-x 2 ulf ulf   4096 Sep 30 17:35 tests
drwxrwxr-x 4 ulf ulf   4096 Sep 30 17:35 _vendor

so there is NO "pkg_resource.py" file inside.

The statement that fails is "from pkg_resources import load_entry_point"

"load_entry_point" is defined in pkg_resources/__init__.py

Somewhere, something decides that
"/usr/lib/python3/dist-packages/pkg_resources.py"
is what we want.

I have a fresh install of Ubuntu-16.04.
This has received the latest package updates.
I have also installed a set of packages I usually install on a machine.
Not much more.

environment lack anything python related.

In one test, I git cloned a fresh copy of "poky".
No layers added.'
Insignificant changes to "local.conf", only changed some directory location

Building the beaglebone, and problem occurs.

If I build from poky "8b4f16a9cbbaf521461f699b7264fac2ac872581" from
Mon Sep 4 11:39:24, I have no problem.

Best Regards
Ulf Samuelsson

then if you do a devshell after do_install, you can check the image 
directory and find the setuptools.egg file, which is what python-native 
uses later, if you unzip that
egg file, you can see that pkg_resources is there, so it was in fact 
packaged on Yocto / by bitbake.


Lastly, you can also do a
$ bitbake bmap-tools-native -c devshell

you can then check that python3-native is in fact executing by doing a

$ which python3

that should point to python3-native (the one you just built)

assuming you got python3-native correctly, execute python3

once in python you can do a


from pkg_resources import load_entry_point # which would import the module

# check where the module is being loaded from
import sys
sys.modules['pkg_resources']

That should print out the location of the imported module, and you can 
see there that infact it was imported from the setuptools.egg file you 
just built before, like thi

Re: [yocto] poky using native pkg_resources module

2017-09-30 Thread Alejandro Hernandez



On 09/30/2017 01:31 PM, Ulf Samuelsson wrote:



Den 2017-09-29 kl. 20:18, skrev Alejandro Hernandez:

Hey Ulf,



On 09/29/2017 04:05 AM, Ulf Samuelsson wrote:

It looks like the pkg_resource module is using the native installation


I cloned poky early september, and could complete a build.

When I clone poky today, the build fails.

As part of "do_image_wic" from image_types_wic.bbclass

A python program in 'bmap-tools-native' is executing

"from pkg_resources import load_entry_point"

grep'ing for pkg_resources in bmap-tools-native reveals:

bmaptool:    from pkg_resources import load_entry_point
easy3_install:    from pkg_resources import load_entry_point
easy_install-3.5:    from pkg_resources import load_entry_point
=
pkg_resources.py is not available inside poky at the moment.

It is (or used to be) part of python-setuptools.


Correct, it was and still is part of setuptools


On Ubuntu, it is present in: 
"http://archive.ubuntu.com/ubuntu/pool/main/p/python-setuptools/python-setuptools_3.3.orig.tar.gz"; 


and used in Ubuntu 14.04 and 16.04 to generate the
"python3-pkg-resource" package.

When python-setuptools is generated by Yocto, it is partly
a class, but the python-setuptools-native recipe downloads
"https://files.pythonhosted.org/packages/source/s/setuptools/setuptools-26.2.7.zip"; 



This should be setuptools-36.2.7 btw


which does not contain "pkg_resources.py"


Its not partly a class, it uses a class to install the package along 
with distutils.


=
When bmaptool imports pkg_resources, it will not find anything in the
yocto build, but will find the native 
"/usr/lib/python3/dist-packages/pkg_resources.py" from


This requires bmap-tools == 3.4, but the native bmap-tools in 
Ubuntu-14.04 is 2.5, and in Ubuntu-16.04 it is 3.2 so the build fails.


Maybe Ubuntu-17.04 supports bmap-tools 3.4 and it will build there,
but using the native pkg_resource is of course not a good idea?


There is something wrong with your build although I am not sure what 
it is, pkg_resources is still part of python(3)-setuptools, in fact 
you can check that in several ways


if you do a devshell on python3-setuptools-native you can see the 
extracted package contains the pkg_resources folder (so it is still 
part of the package)

>
I do: "bitbake  -c devshell python3-setuptools-native"
A terminal windows opens in 
"tmp/work/x86_64-linux/python3-setuptools-native/36.2.7-r0/setuptools-36.2.7"


ls
bootstrap.py  easy_install.py  PKG-INFO   setuptools
build launcher.c   pkg_resources setuptools.egg-info
CHANGES.rst   LICENSE  pytest.ini tests
conftest.py   MANIFEST.in  README.rst tox.ini
dist  msvc-build-launcher.cmd  setup.cfg
docs  pavement.py  setup.py

so the "pkg_resources" folder is present.

cd pkg_resources ; ls -l
-rw-r--r-- 1 ulf ulf  12211 Aug  2 00:40 api_tests.txt
drwxrwxr-x 3 ulf ulf   4096 Sep 30 17:35 extern
-rw-r--r-- 1 ulf ulf 104650 Aug  2 00:40 __init__.py
-rw-r--r-- 1 ulf ulf    600 Aug  2 00:40 py31compat.py
drwxrwxr-x 2 ulf ulf   4096 Sep 30 17:35 __pycache__
drwxrwxr-x 2 ulf ulf   4096 Sep 30 17:35 tests
drwxrwxr-x 4 ulf ulf   4096 Sep 30 17:35 _vendor

so there is NO "pkg_resource.py" file inside.

The statement that fails is "from pkg_resources import load_entry_point"

"load_entry_point" is defined in pkg_resources/__init__.py

Somewhere, something decides that
"/usr/lib/python3/dist-packages/pkg_resources.py"
is what we want.

I have a fresh install of Ubuntu-16.04.
This has received the latest package updates.
I have also installed a set of packages I usually install on a machine.
Not much more.

environment lack anything python related.

In one test, I git cloned a fresh copy of "poky".
No layers added.'
Insignificant changes to "local.conf", only changed some directory 
location


Building the beaglebone, and problem occurs.

If I build from poky "8b4f16a9cbbaf521461f699b7264fac2ac872581" from
Mon Sep 4 11:39:24, I have no problem.

Best Regards
Ulf Samuelsson



Correct, please submit a bug on https://bugzilla.yoctoproject.org

This is a bmap-tools-native bug failing to load pkg_resources correctly, 
most likely caused by  the switch to python3

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=04ef46e2b2b753321408db02a2df3753b85ac0dd

I will take a look at this once the bug is filled.

Thanks!

Alejandro




then if you do a devshell after do_install, you can check the image 
directory and find the setuptools.egg file, which is what 
python-native uses later, if you unzip that
egg file, you can see that pkg_resources is there, so it was in fact 
packaged on Yocto / by bitbake.


Lastly, you can also do a
$ bitbake bmap-tools-native -c devshell

you can then check that python3-native is in fact executing by doing a

$ which python3

that should point to python3-native (the one you just built)

assuming y

Re: [yocto] poky using native pkg_resources module

2017-09-30 Thread Alejandro Hernandez

Hey Ulf,


Yes, the problem is that the bmap-tools script is trying to be executed 
by python3 (HOST) but it should be


executed by the native python3 we just built, so fixing the shebang on 
the script should do it,


please file the bug and email me the bug number so I can take care of it.


Alejandro | aehs29


On 09/30/2017 01:31 PM, Ulf Samuelsson wrote:



Den 2017-09-29 kl. 20:18, skrev Alejandro Hernandez:

Hey Ulf,



On 09/29/2017 04:05 AM, Ulf Samuelsson wrote:

It looks like the pkg_resource module is using the native installation


I cloned poky early september, and could complete a build.

When I clone poky today, the build fails.

As part of "do_image_wic" from image_types_wic.bbclass

A python program in 'bmap-tools-native' is executing

"from pkg_resources import load_entry_point"

grep'ing for pkg_resources in bmap-tools-native reveals:

bmaptool:    from pkg_resources import load_entry_point
easy3_install:    from pkg_resources import load_entry_point
easy_install-3.5:    from pkg_resources import load_entry_point
=
pkg_resources.py is not available inside poky at the moment.

It is (or used to be) part of python-setuptools.


Correct, it was and still is part of setuptools


On Ubuntu, it is present in: 
"http://archive.ubuntu.com/ubuntu/pool/main/p/python-setuptools/python-setuptools_3.3.orig.tar.gz"; 


and used in Ubuntu 14.04 and 16.04 to generate the
"python3-pkg-resource" package.

When python-setuptools is generated by Yocto, it is partly
a class, but the python-setuptools-native recipe downloads
"https://files.pythonhosted.org/packages/source/s/setuptools/setuptools-26.2.7.zip"; 



This should be setuptools-36.2.7 btw


which does not contain "pkg_resources.py"


Its not partly a class, it uses a class to install the package along 
with distutils.


=
When bmaptool imports pkg_resources, it will not find anything in the
yocto build, but will find the native 
"/usr/lib/python3/dist-packages/pkg_resources.py" from


This requires bmap-tools == 3.4, but the native bmap-tools in 
Ubuntu-14.04 is 2.5, and in Ubuntu-16.04 it is 3.2 so the build fails.


Maybe Ubuntu-17.04 supports bmap-tools 3.4 and it will build there,
but using the native pkg_resource is of course not a good idea?


There is something wrong with your build although I am not sure what 
it is, pkg_resources is still part of python(3)-setuptools, in fact 
you can check that in several ways


if you do a devshell on python3-setuptools-native you can see the 
extracted package contains the pkg_resources folder (so it is still 
part of the package)

>
I do: "bitbake  -c devshell python3-setuptools-native"
A terminal windows opens in 
"tmp/work/x86_64-linux/python3-setuptools-native/36.2.7-r0/setuptools-36.2.7"


ls
bootstrap.py  easy_install.py  PKG-INFO   setuptools
build launcher.c   pkg_resources setuptools.egg-info
CHANGES.rst   LICENSE  pytest.ini tests
conftest.py   MANIFEST.in  README.rst tox.ini
dist  msvc-build-launcher.cmd  setup.cfg
docs  pavement.py  setup.py

so the "pkg_resources" folder is present.

cd pkg_resources ; ls -l
-rw-r--r-- 1 ulf ulf  12211 Aug  2 00:40 api_tests.txt
drwxrwxr-x 3 ulf ulf   4096 Sep 30 17:35 extern
-rw-r--r-- 1 ulf ulf 104650 Aug  2 00:40 __init__.py
-rw-r--r-- 1 ulf ulf    600 Aug  2 00:40 py31compat.py
drwxrwxr-x 2 ulf ulf   4096 Sep 30 17:35 __pycache__
drwxrwxr-x 2 ulf ulf   4096 Sep 30 17:35 tests
drwxrwxr-x 4 ulf ulf   4096 Sep 30 17:35 _vendor

so there is NO "pkg_resource.py" file inside.

The statement that fails is "from pkg_resources import load_entry_point"

"load_entry_point" is defined in pkg_resources/__init__.py

Somewhere, something decides that
"/usr/lib/python3/dist-packages/pkg_resources.py"
is what we want.

I have a fresh install of Ubuntu-16.04.
This has received the latest package updates.
I have also installed a set of packages I usually install on a machine.
Not much more.

environment lack anything python related.

In one test, I git cloned a fresh copy of "poky".
No layers added.'
Insignificant changes to "local.conf", only changed some directory 
location


Building the beaglebone, and problem occurs.

If I build from poky "8b4f16a9cbbaf521461f699b7264fac2ac872581" from
Mon Sep 4 11:39:24, I have no problem.

Best Regards
Ulf Samuelsson

then if you do a devshell after do_install, you can check the image 
directory and find the setuptools.egg file, which is what 
python-native uses later, if you unzip that
egg file, you can see that pkg_resources is there, so it was in fact 
packaged on Yocto / by bitbake.


Lastly, you can also do a
$ bitbake bmap-tools-native -c devshell

you can then check that python3-native is in fact executing by doing a

$ which python3

that should point to python3-native (the one you just built)

assuming you got python3-native correctly, execute python3

Re: [yocto] poky using native pkg_resources module

2017-09-30 Thread Ulf Samuelsson



Den 2017-09-30 kl. 21:32, skrev Alejandro Hernandez:

Hey Ulf,


Yes, the problem is that the bmap-tools script is trying to be executed 
by python3 (HOST) but it should be


executed by the native python3 we just built, so fixing the shebang on 
the script should do it,


please file the bug and email me the bug number so I can take care of it.


https://bugzilla.yoctoproject.org/show_bug.cgi?id=12148

BR
Ulf




Alejandro | aehs29


On 09/30/2017 01:31 PM, Ulf Samuelsson wrote:



Den 2017-09-29 kl. 20:18, skrev Alejandro Hernandez:

Hey Ulf,



On 09/29/2017 04:05 AM, Ulf Samuelsson wrote:

It looks like the pkg_resource module is using the native installation


I cloned poky early september, and could complete a build.

When I clone poky today, the build fails.

As part of "do_image_wic" from image_types_wic.bbclass

A python program in 'bmap-tools-native' is executing

"from pkg_resources import load_entry_point"

grep'ing for pkg_resources in bmap-tools-native reveals:

bmaptool:    from pkg_resources import load_entry_point
easy3_install:    from pkg_resources import load_entry_point
easy_install-3.5:    from pkg_resources import load_entry_point
=
pkg_resources.py is not available inside poky at the moment.

It is (or used to be) part of python-setuptools.


Correct, it was and still is part of setuptools


On Ubuntu, it is present in: 
"http://archive.ubuntu.com/ubuntu/pool/main/p/python-setuptools/python-setuptools_3.3.orig.tar.gz"; 


and used in Ubuntu 14.04 and 16.04 to generate the
"python3-pkg-resource" package.

When python-setuptools is generated by Yocto, it is partly
a class, but the python-setuptools-native recipe downloads
"https://files.pythonhosted.org/packages/source/s/setuptools/setuptools-26.2.7.zip"; 



This should be setuptools-36.2.7 btw


which does not contain "pkg_resources.py"


Its not partly a class, it uses a class to install the package along 
with distutils.


=
When bmaptool imports pkg_resources, it will not find anything in the
yocto build, but will find the native 
"/usr/lib/python3/dist-packages/pkg_resources.py" from


This requires bmap-tools == 3.4, but the native bmap-tools in 
Ubuntu-14.04 is 2.5, and in Ubuntu-16.04 it is 3.2 so the build fails.


Maybe Ubuntu-17.04 supports bmap-tools 3.4 and it will build there,
but using the native pkg_resource is of course not a good idea?


There is something wrong with your build although I am not sure what 
it is, pkg_resources is still part of python(3)-setuptools, in fact 
you can check that in several ways


if you do a devshell on python3-setuptools-native you can see the 
extracted package contains the pkg_resources folder (so it is still 
part of the package)

>
I do: "bitbake  -c devshell python3-setuptools-native"
A terminal windows opens in 
"tmp/work/x86_64-linux/python3-setuptools-native/36.2.7-r0/setuptools-36.2.7" 



ls
bootstrap.py  easy_install.py  PKG-INFO   setuptools
build launcher.c   pkg_resources setuptools.egg-info
CHANGES.rst   LICENSE  pytest.ini tests
conftest.py   MANIFEST.in  README.rst tox.ini
dist  msvc-build-launcher.cmd  setup.cfg
docs  pavement.py  setup.py

so the "pkg_resources" folder is present.

cd pkg_resources ; ls -l
-rw-r--r-- 1 ulf ulf  12211 Aug  2 00:40 api_tests.txt
drwxrwxr-x 3 ulf ulf   4096 Sep 30 17:35 extern
-rw-r--r-- 1 ulf ulf 104650 Aug  2 00:40 __init__.py
-rw-r--r-- 1 ulf ulf    600 Aug  2 00:40 py31compat.py
drwxrwxr-x 2 ulf ulf   4096 Sep 30 17:35 __pycache__
drwxrwxr-x 2 ulf ulf   4096 Sep 30 17:35 tests
drwxrwxr-x 4 ulf ulf   4096 Sep 30 17:35 _vendor

so there is NO "pkg_resource.py" file inside.

The statement that fails is "from pkg_resources import load_entry_point"

"load_entry_point" is defined in pkg_resources/__init__.py

Somewhere, something decides that
"/usr/lib/python3/dist-packages/pkg_resources.py"
is what we want.

I have a fresh install of Ubuntu-16.04.
This has received the latest package updates.
I have also installed a set of packages I usually install on a machine.
Not much more.

environment lack anything python related.

In one test, I git cloned a fresh copy of "poky".
No layers added.'
Insignificant changes to "local.conf", only changed some directory 
location


Building the beaglebone, and problem occurs.

If I build from poky "8b4f16a9cbbaf521461f699b7264fac2ac872581" from
Mon Sep 4 11:39:24, I have no problem.

Best Regards
Ulf Samuelsson

then if you do a devshell after do_install, you can check the image 
directory and find the setuptools.egg file, which is what 
python-native uses later, if you unzip that
egg file, you can see that pkg_resources is there, so it was in fact 
packaged on Yocto / by bitbake.


Lastly, you can also do a
$ bitbake bmap-tools-native -c devshell

you can then check that python3-native is in fact executing by doing a

$ which pyth