Re: [yocto] SDK Problems for SuperH 4

2016-10-07 Thread Martin Townsend 
On Fri, Oct 7, 2016 at 5:32 AM, Khem Raj  wrote:
>
>> On Oct 6, 2016, at 12:09 PM, Martin Townsend  wrote:
>>
>> On Thu, Oct 6, 2016 at 7:57 PM, Martin Townsend  
>> wrote:
>>> Hi,
>>>
>>> We have a poky sato distributions successfully built and running for
>>> our SuperH 4 processor board.  The problem is the SDK that is built
>>> when using -cpopulate_sdk doesn't.
>>>
>>> The simple Makefile:
>>> hellomake: HelloWorld.c
>>> $(CC) -o HelloWorld HelloWorld.c
>>
>> Hit send before I had chance to finish :)
>>
>> ignore the missing tab but when trying to compile (after sourcing the
>> SDK environment) I get
>> sh4-poky-linux-gcc  -ml -m4
>> --sysroot=/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux -o HelloWorld
>> HelloWorld.o
>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>> cannot find crt1.o: No such file or directory
>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>> cannot find crti.o: No such file or directory
>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>> cannot find crtbegin.o: No such file or directory
>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>> cannot find -lgcc
>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>> cannot find -lgcc_s
>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>> cannot find -lc
>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>> cannot find -lgcc
>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>> cannot find -lgcc_s
>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>> cannot find crtend.o: No such file or directory
>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>> cannot find crtn.o: No such file or directory
>>
>> So I tried the compiler and sysroot from the build
>> CC=/ws/vms/vms-rsc-yocto-build/build_vms/tmp/sysroots/x86_64-linux/usr/bin/sh4-poky-linux/sh4-poky-linux-gcc
>> SYSROOT=/ws/vms/vms-rsc-yocto-build/build_vms/tmp/sysroots/sh7760
>>
>> hellomake: HelloWorld.c
>> $(CC) --sysroot=$(SYSROOT) -o HelloWorld HelloWorld.c
>>
>> and it compiles fine.
>>
>> So I ran $CC -print-search-dirs
>> install: 
>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/
>> programs: 
>> =/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/../../../../../sh4-poky-linux/bin/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/../../../../../sh4-poky-linux/bin/
>> libraries: 
>> =/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/../../../../../sh4-poky-linux/lib/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/../../../../../sh4-poky-linux/lib/:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/m4/lib/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/m4/lib/:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/m4/usr/lib/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/m4/usr/lib/
>>
>> Then I checked that these files exist:
>> martin@martin-ubuntu-pconcepts:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux$
>> find $SDKTARGETSYSROOT -name "crt*"
>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/crtn.o
>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/crti.o
>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/crt1.o
>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/sh4-poky-linux/4.9.3/crtendS.o
>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/sh4-poky-linux/4.9.3/crtend.o
>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/sh4-poky-linux/4.9.3/crtbeginT.o
>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/sh4-poky-linux/4.9.3/crtbeginS.o
>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/sh4-poky-linux/4.9.3/crtbegin.o
>>
>> Putting the library search directories on a sep

Re: [yocto] SDK Problems for SuperH 4

2016-10-07 Thread Martin Townsend 
On Fri, Oct 7, 2016 at 8:46 AM, Martin Townsend  wrote:
> On Fri, Oct 7, 2016 at 5:32 AM, Khem Raj  wrote:
>>
>>> On Oct 6, 2016, at 12:09 PM, Martin Townsend  
>>> wrote:
>>>
>>> On Thu, Oct 6, 2016 at 7:57 PM, Martin Townsend  
>>> wrote:
 Hi,

 We have a poky sato distributions successfully built and running for
 our SuperH 4 processor board.  The problem is the SDK that is built
 when using -cpopulate_sdk doesn't.

 The simple Makefile:
 hellomake: HelloWorld.c
 $(CC) -o HelloWorld HelloWorld.c
>>>
>>> Hit send before I had chance to finish :)
>>>
>>> ignore the missing tab but when trying to compile (after sourcing the
>>> SDK environment) I get
>>> sh4-poky-linux-gcc  -ml -m4
>>> --sysroot=/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux -o HelloWorld
>>> HelloWorld.o
>>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>>> cannot find crt1.o: No such file or directory
>>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>>> cannot find crti.o: No such file or directory
>>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>>> cannot find crtbegin.o: No such file or directory
>>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>>> cannot find -lgcc
>>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>>> cannot find -lgcc_s
>>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>>> cannot find -lc
>>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>>> cannot find -lgcc
>>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>>> cannot find -lgcc_s
>>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>>> cannot find crtend.o: No such file or directory
>>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
>>> cannot find crtn.o: No such file or directory
>>>
>>> So I tried the compiler and sysroot from the build
>>> CC=/ws/vms/vms-rsc-yocto-build/build_vms/tmp/sysroots/x86_64-linux/usr/bin/sh4-poky-linux/sh4-poky-linux-gcc
>>> SYSROOT=/ws/vms/vms-rsc-yocto-build/build_vms/tmp/sysroots/sh7760
>>>
>>> hellomake: HelloWorld.c
>>> $(CC) --sysroot=$(SYSROOT) -o HelloWorld HelloWorld.c
>>>
>>> and it compiles fine.
>>>
>>> So I ran $CC -print-search-dirs
>>> install: 
>>> /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/
>>> programs: 
>>> =/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/../../../../../sh4-poky-linux/bin/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/../../../../../sh4-poky-linux/bin/
>>> libraries: 
>>> =/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/../../../../../sh4-poky-linux/lib/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/../../../../../sh4-poky-linux/lib/:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/m4/lib/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/m4/lib/:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/m4/usr/lib/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/m4/usr/lib/
>>>
>>> Then I checked that these files exist:
>>> martin@martin-ubuntu-pconcepts:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux$
>>> find $SDKTARGETSYSROOT -name "crt*"
>>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/crtn.o
>>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/crti.o
>>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/crt1.o
>>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/sh4-poky-linux/4.9.3/crtendS.o
>>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/sh4-poky-linux/4.9.3/crtend.o
>>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/sh4-poky-linux/4.9.3/crtbeginT.o
>>> /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/sh4-poky-linux/4.9.3/crtbeginS.o
>>> /o

Re: [yocto] which meta-intel woudl build a kernel 4.8 (solved)

2016-10-07 Thread Dominig ar Foll (Intel Open Source) 


Le 06/10/2016 à 20:19, Khem Raj a écrit :
>
> Use poky or oe-core master with master of meta-intel is best shot
>
>
thanks updating as well to 'master' my oe-core and poky has fixed that
issue.

-- 
Dominig ar Foll
Senior Software Architect
Intel Open Source Technology Centre

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] SDK Problems for SuperH 4

2016-10-07 Thread Khem Raj 

> On Oct 7, 2016, at 12:58 AM, Martin Townsend  wrote:
> 
> On Fri, Oct 7, 2016 at 8:46 AM, Martin Townsend  
> wrote:
>> On Fri, Oct 7, 2016 at 5:32 AM, Khem Raj  wrote:
>>> 
 On Oct 6, 2016, at 12:09 PM, Martin Townsend  
 wrote:
 
 On Thu, Oct 6, 2016 at 7:57 PM, Martin Townsend  
 wrote:
> Hi,
> 
> We have a poky sato distributions successfully built and running for
> our SuperH 4 processor board.  The problem is the SDK that is built
> when using -cpopulate_sdk doesn't.
> 
> The simple Makefile:
> hellomake: HelloWorld.c
> $(CC) -o HelloWorld HelloWorld.c
 
 Hit send before I had chance to finish :)
 
 ignore the missing tab but when trying to compile (after sourcing the
 SDK environment) I get
 sh4-poky-linux-gcc  -ml -m4
 --sysroot=/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux -o HelloWorld
 HelloWorld.o
 /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
 cannot find crt1.o: No such file or directory
 /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
 cannot find crti.o: No such file or directory
 /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
 cannot find crtbegin.o: No such file or directory
 /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
 cannot find -lgcc
 /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
 cannot find -lgcc_s
 /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
 cannot find -lc
 /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
 cannot find -lgcc
 /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
 cannot find -lgcc_s
 /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
 cannot find crtend.o: No such file or directory
 /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/real-ld:
 cannot find crtn.o: No such file or directory
 
 So I tried the compiler and sysroot from the build
 CC=/ws/vms/vms-rsc-yocto-build/build_vms/tmp/sysroots/x86_64-linux/usr/bin/sh4-poky-linux/sh4-poky-linux-gcc
 SYSROOT=/ws/vms/vms-rsc-yocto-build/build_vms/tmp/sysroots/sh7760
 
 hellomake: HelloWorld.c
 $(CC) --sysroot=$(SYSROOT) -o HelloWorld HelloWorld.c
 
 and it compiles fine.
 
 So I ran $CC -print-search-dirs
 install: 
 /opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/
 programs: 
 =/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/libexec/sh4-poky-linux/gcc/sh4-poky-linux/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/../../../../../sh4-poky-linux/bin/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/../../../../../sh4-poky-linux/bin/
 libraries: 
 =/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/../../../../../sh4-poky-linux/lib/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/x86_64-pokysdk-linux/usr/lib/sh4-poky-linux/gcc/sh4-poky-linux/4.9.3/../../../../../sh4-poky-linux/lib/:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/m4/lib/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/m4/lib/:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/m4/usr/lib/sh4-poky-linux/4.9.3/:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/m4/usr/lib/
 
 Then I checked that these files exist:
 martin@martin-ubuntu-pconcepts:/opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux$
 find $SDKTARGETSYSROOT -name "crt*"
 /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/crtn.o
 /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/crti.o
 /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/crt1.o
 /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/sh4-poky-linux/4.9.3/crtendS.o
 /opt/sdk/vms/2.0.2/sysroots/sh4-poky-linux/usr/lib/sh4-poky-linux/4.9.3/crtend.o
 /opt/sdk/vms/2.0.2/sysroots/sh4-

[yocto] Yocto Project Status WW41

2016-10-07 Thread Jolley, Stephen K 
Current Dev Position: YP 2.2 M4

Next Deadline: YP 2.2 M4 which will be Oct. 3rd (5:00pm GMT)


SWAT team rotation: Armin -> Saul

https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team


Key Status/Updates:

*The 4.8 kernel was released and we've updated to the release version. 
Sadly they broke the userspace headers late in the cycle so we had to work 
around issues there and whilst the qemu machines are updated, we're still 
waiting on the hardware reference BSPs. This should be completed by Monday.

*Based on this, the first rc build of 2.2 should therefore be on Monday.

*We have fixed a good number of medium+ bugs and the list of bugs we're 
targeting for the release is starting to diminish.

*There are still a number of worrying issues such as eSDK performance 
questions, an sdk toolchain issue and various other issues we'd ideally like to 
fix for the reason, several of which have patches in testing.

*The YP 2.2 release branches ("morty") have been created.

*The defect tracking metric dropped again this week and we're now just 
below the level we reached before the 2.1 release. We've not managed to reduce 
the bug defect density and open bugs counts like this for a while so this is 
good news!

*We are starting to defer more invasive changes to be post 2.2 in the 
2.3 timeframe.


Key YP 2.2 Dates:

*YP 2.2 M4 cut off would be: 10/3/16

*YP 2.2 M4 release would be: 10/28/16


Tracking Metrics:

WDD 2380 (last week 2451)

(https://wiki.yoctoproject.org/charts/combo.html)


Key Status Links for YP:

https://wiki.yoctoproject.org/wiki/Yocto_Project_v2.2_Status

https://wiki.yoctoproject.org/wiki/Yocto_2.2_Schedule

https://wiki.yoctoproject.org/wiki/Yocto_2.2_Features


[If anyone has suggestions for other information you'd like to see on this 
weekly status update, let us know!]

Thanks,

Stephen K. Jolley
Yocto Project Program Manager
INTEL, MS JF1-255, 2111 N.E. 25th Avenue, Hillsboro, OR 97124
*   Work Telephone:(503) 712-0534
*Cell:   (208) 244-4460
* Email:stephen.k.jol...@intel.com

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [layerindex-web][PATCH v2 02/12] layerindex/tools/import_layer.py: Sanitize layer name.

2016-10-07 Thread Liam R. Howlett 
Django will produce a cryptic error message if layers are added with
invalid names.  Sanitize the layer names when trying to add them.

Signed-off-by: Liam R. Howlett 
---
 layerindex/tools/import_layer.py | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/layerindex/tools/import_layer.py b/layerindex/tools/import_layer.py
index 9b5da22..fefef0c 100755
--- a/layerindex/tools/import_layer.py
+++ b/layerindex/tools/import_layer.py
@@ -181,6 +181,8 @@ def get_github_layerinfo(layer_url, username = None, 
password = None):
 
 
 def main():
+valid_layer_name = re.compile('[-\w]+$')
+
 parser = optparse.OptionParser(
 usage = """
 %prog [options]  [name]""")
@@ -222,6 +224,10 @@ def main():
 if layer_name.endswith('.git'):
 layer_name = layer_name[:-4]
 
+if not valid_layer_name.match(layer_name):
+logger.error('Invalid layer name "%s" -  Layer name can only include 
letters, numbers and dashes.', layer_name)
+sys.exit(1)
+
 if options.github_auth:
 if not ':' in options.github_auth:
 logger.error('--github-auth value must be specified as 
username:password')
-- 
1.9.1

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [layerindex-web][PATCH v2 00/12] Add Distro, dependency and recommends detection, and import_project

2016-10-07 Thread Liam R. Howlett 
This set of patches adds a number of features to the layerindex code:
 -  Adds the ability to set the actual-branch to import_layer.
 -  Adds distro to the database and web interface.
 -  Adds collection & version information to layerbranch.
 -  Uses layer name and collection name to determine dependencies and
recommends.
 -  Adds import_project to simplify pulling all layers and openembedded-core
into a layer index.

Liam R. Howlett (11):
  import_layer: Add --actual-branch option
  layerindex/tools/import_layer.py: Sanitize layer name.
  layerindex/tools/import_layer.py: Avoid failing if there is any layer
to add.
  layerindex/utils: Update runcmd to decode binary strings to strings.
  layerindex: Add distro to web interface and model.
  layerindex/tools/import_project: Add import_project
  layerindex/recipeparse.py: refactor setup_tinfoil,
checkout_layer_branch, parse_layer_conf to utils.py
  layerindex: Detect dependencies from layer.conf files
  layerindex: Add collection and version to layerbranch
  layerindexer: Add layer recommends support
  recipeparse: remove unnecessary else statement.

Mark Hatle (1):
  layerindex/update_layer.py: Preserve the recipedependency files

 layerindex/admin.py|  11 +++
 layerindex/bulkchange.py   |   4 +-
 layerindex/layerconfparse.py   |  50 +++
 layerindex/models.py   |  23 +
 layerindex/recipeparse.py  |  53 +++
 layerindex/restviews.py|  10 ++-
 layerindex/tools/import_classic.py |   2 +
 layerindex/tools/import_layer.py   |  47 +-
 layerindex/tools/import_project.py | 180 +
 layerindex/update.py   |  42 -
 layerindex/update_layer.py |  87 --
 layerindex/urls.py |   3 +
 layerindex/urls_branch.py  |   6 +-
 layerindex/utils.py| 159 +++-
 layerindex/views.py|  29 +-
 templates/layerindex/detail.html   |  55 +---
 templates/layerindex/distros.html  |  76 
 templates/layerindex/layers.html   |   1 +
 templates/layerindex/machines.html |   1 +
 templates/layerindex/recipes.html  |   1 +
 20 files changed, 767 insertions(+), 73 deletions(-)
 create mode 100644 layerindex/layerconfparse.py
 create mode 100755 layerindex/tools/import_project.py
 create mode 100644 templates/layerindex/distros.html

-- 
1.9.1

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [layerindex-web][PATCH v2 03/12] layerindex/tools/import_layer.py: Avoid failing if there is any layer to add.

2016-10-07 Thread Liam R. Howlett 
Subdirectories are scanned when adding layers.  If any of the
subdirectories or root directory layers already exist in the database,
then the addition fails.  This changes that behaviour to report the
failure as a warning and remove it from the list.  That way, if a repo
has a new layer added it can be rescanned without issue.  Layers being
rescanned are checked against the vcs_url to ensure there is not a name
collision.  A name collision without the same vcs_url will still produce
a hard failure.

Note that multiple layers with the same vcs_url are supported in the
error reporting even though this should never happen.

Signed-off-by: Liam R. Howlett 
---
 layerindex/tools/import_layer.py | 20 ++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/layerindex/tools/import_layer.py b/layerindex/tools/import_layer.py
index fefef0c..184c5cc 100755
--- a/layerindex/tools/import_layer.py
+++ b/layerindex/tools/import_layer.py
@@ -334,8 +334,20 @@ def main():
 else:
 subdir = ''
 if LayerItem.objects.filter(name=layer.name).exists():
-logger.error('A layer named "%s" already exists in the 
database' % layer_name)
-sys.exit(1)
+if 
LayerItem.objects.filter(name=layer.name).exclude(vcs_url=layer.vcs_url).exists():
+conflict_list = 
LayerItem.objects.filter(name=layer.name).exclude(vcs_url=layer.vcs_url)
+conflict_list_urls = []
+for conflict in conflict_list:
+conflict_list_urls.append(conflict.vcs_url)
+cln = ', '.join(conflict_list_urls)
+logger.error('A layer named "%s" already exists in the 
database.  Possible name collision with %s.vcs_url = %s' % (layer.name, 
layer.name, cln))
+sys.exit(1)
+else:
+logger.info('The layer named "%s" already exists in 
the database. Skipping this layer with same vcs_url' % layer.name)
+layer_paths = [x for x in layer_paths if x != layerdir]
+continue
+
+
 
 logger.info('Creating layer %s' % layer.name)
 # Guess layer type
@@ -411,6 +423,10 @@ def main():
 
 layer.save()
 
+if not layer_paths:
+logger.error('No layers added.')
+sys.exit(1);
+
 if options.dryrun:
 raise DryRunRollbackException()
 except DryRunRollbackException:
-- 
1.9.1

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [layerindex-web][PATCH v2 01/12] import_layer: Add --actual-branch option

2016-10-07 Thread Liam R. Howlett 
Allow users to set actual-branch from the command line import of layers.

Signed-off-by: Liam R. Howlett 
---
 layerindex/tools/import_layer.py | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/layerindex/tools/import_layer.py b/layerindex/tools/import_layer.py
index 0521e1c..9b5da22 100755
--- a/layerindex/tools/import_layer.py
+++ b/layerindex/tools/import_layer.py
@@ -200,6 +200,9 @@ def main():
 parser.add_option("-q", "--quiet",
 help = "Hide all output except error messages",
 action="store_const", const=logging.ERROR, dest="loglevel")
+parser.add_option("-a", "--actual-branch",
+help = "Set actual branch",
+action="store", dest="actual_branch")
 
 options, args = parser.parse_args(sys.argv)
 
@@ -273,10 +276,13 @@ def main():
 logger.error("Fetch failed: %s" % str(e))
 sys.exit(1)
 
-actual_branch = ''
+actual_branch = 'master'
+if (options.actual_branch):
+actual_branch = options.actual_branch
 try:
-out = utils.runcmd("git checkout origin/master", repodir, 
logger=logger)
+out = utils.runcmd("git checkout origin/%s" % actual_branch, 
repodir, logger=logger)
 except subprocess.CalledProcessError:
+actual_branch = None
 branches = utils.runcmd("git branch -r", repodir, 
logger=logger)
 for line in branches.splitlines():
 if 'origin/HEAD ->' in line:
-- 
1.9.1

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [layerindex-web][PATCH v2 05/12] layerindex: Add distro to web interface and model.

2016-10-07 Thread Liam R. Howlett 
Add the distros to the index.  This looks a lot like the machines
and allows users to search for a particular distro.

Signed-off-by: Liam R. Howlett 
---
 layerindex/admin.py| 11 ++
 layerindex/models.py   | 14 +++
 layerindex/recipeparse.py  |  5 +++
 layerindex/restviews.py| 10 -
 layerindex/tools/import_classic.py |  2 +
 layerindex/update_layer.py | 46 ++-
 layerindex/urls.py |  3 ++
 layerindex/urls_branch.py  |  6 ++-
 layerindex/views.py| 29 ++-
 templates/layerindex/detail.html   | 23 
 templates/layerindex/distros.html  | 76 ++
 templates/layerindex/layers.html   |  1 +
 templates/layerindex/machines.html |  1 +
 templates/layerindex/recipes.html  |  1 +
 14 files changed, 224 insertions(+), 4 deletions(-)
 create mode 100644 templates/layerindex/distros.html

diff --git a/layerindex/admin.py b/layerindex/admin.py
index accb954..7339ce1 100644
--- a/layerindex/admin.py
+++ b/layerindex/admin.py
@@ -75,6 +75,16 @@ class MachineAdmin(admin.ModelAdmin):
 def has_delete_permission(self, request, obj=None):
 return False
 
+class DistroAdmin(admin.ModelAdmin):
+search_fields = ['name']
+list_filter = ['layerbranch__layer__name', 'layerbranch__branch__name']
+readonly_fields = Distro._meta.get_all_field_names()
+def has_add_permission(self, request, obj=None):
+return False
+def has_delete_permission(self, request, obj=None):
+return False
+
+
 class BBAppendAdmin(admin.ModelAdmin):
 search_fields = ['filename']
 list_filter = ['layerbranch__layer__name', 'layerbranch__branch__name']
@@ -111,6 +121,7 @@ admin.site.register(LayerNote, LayerNoteAdmin)
 admin.site.register(Recipe, RecipeAdmin)
 admin.site.register(RecipeFileDependency)
 admin.site.register(Machine, MachineAdmin)
+admin.site.register(Distro, DistroAdmin)
 admin.site.register(BBAppend, BBAppendAdmin)
 admin.site.register(BBClass, BBClassAdmin)
 admin.site.register(RecipeChangeset, RecipeChangesetAdmin)
diff --git a/layerindex/models.py b/layerindex/models.py
index 3fd16d4..2343ba7 100644
--- a/layerindex/models.py
+++ b/layerindex/models.py
@@ -361,6 +361,20 @@ class Machine(models.Model):
 def __str__(self):
 return '%s (%s)' % (self.name, self.layerbranch.layer.name)
 
+class Distro(models.Model):
+layerbranch = models.ForeignKey(LayerBranch)
+name = models.CharField(max_length=255)
+description = models.CharField(max_length=255)
+
+updated = models.DateTimeField(auto_now=True)
+
+def vcs_web_url(self):
+url = self.layerbranch.file_url(os.path.join('conf/distro/%s.conf' % 
self.name))
+return url or ''
+
+def __str__(self):
+return '%s (%s)' % (self.name, self.layerbranch.layer.name)
+
 
 class BBAppend(models.Model):
 layerbranch = models.ForeignKey(LayerBranch)
diff --git a/layerindex/recipeparse.py b/layerindex/recipeparse.py
index 7f995ec..a9cecfc 100644
--- a/layerindex/recipeparse.py
+++ b/layerindex/recipeparse.py
@@ -152,6 +152,7 @@ def get_var_files(fn, varlist, d):
 return varfiles
 
 machine_conf_re = re.compile(r'conf/machine/([^/.]*).conf$')
+distro_conf_re = re.compile(r'conf/distro/([^/.]*).conf$')
 bbclass_re = re.compile(r'classes/([^/.]*).bbclass$')
 def detect_file_type(path, subdir_start):
 typename = None
@@ -171,6 +172,10 @@ def detect_file_type(path, subdir_start):
 if res:
 typename = 'bbclass'
 return (typename, None, res.group(1))
+res = distro_conf_re.match(subpath)
+if res:
+typename = 'distro'
+return (typename, None, res.group(1))
 
 if typename == 'recipe' or typename == 'bbappend':
 if subdir_start:
diff --git a/layerindex/restviews.py b/layerindex/restviews.py
index b33d3d1..57f1552 100644
--- a/layerindex/restviews.py
+++ b/layerindex/restviews.py
@@ -1,4 +1,4 @@
-from layerindex.models import Branch, LayerItem, LayerNote, LayerBranch, 
LayerDependency, Recipe, Machine
+from layerindex.models import Branch, LayerItem, LayerNote, LayerBranch, 
LayerDependency, Recipe, Machine, Distro
 from rest_framework import viewsets, serializers
 from layerindex.querysethelper import params_to_queryset, get_search_tuple
 
@@ -56,3 +56,11 @@ class MachineSerializer(serializers.ModelSerializer):
 class MachineViewSet(ParametricSearchableModelViewSet):
 queryset = Machine.objects.all()
 serializer_class = MachineSerializer
+
+class DistroSerializer(serializers.ModelSerializer):
+class Meta:
+model = Distro
+
+class DistroViewSet(ParametricSearchableModelViewSet):
+queryset = Distro.objects.all()
+serializer_class = DistroSerializer
diff --git a/layerindex/tools/import_classic.py 
b/layerindex/tools/import_classic.py
index 45ccaa9..80c49af 100755
--- a/layerindex/tools/import

[yocto] [layerindex-web][PATCH v2 08/12] layerindex: Detect dependencies from layer.conf files

2016-10-07 Thread Liam R. Howlett 
Read dependencies from layer.conf and try to create the LayerDependency
entry by looking up the correct database object.  Dependencies are found
by layer name only - no collection support.  layer.conf parsing is
handled by the bitbake code.

Once all layers are added, the dependencies have to be rechecked in case
the layers are not added in order.

Signed-off-by: Liam R. Howlett 
---
 layerindex/layerconfparse.py | 50 
 layerindex/tools/import_layer.py | 11 ++
 layerindex/update.py | 41 +++-
 layerindex/update_layer.py   | 20 ++
 layerindex/utils.py  | 82 
 5 files changed, 196 insertions(+), 8 deletions(-)
 create mode 100644 layerindex/layerconfparse.py

diff --git a/layerindex/layerconfparse.py b/layerindex/layerconfparse.py
new file mode 100644
index 000..3a453bd
--- /dev/null
+++ b/layerindex/layerconfparse.py
@@ -0,0 +1,50 @@
+# Utility functions for parsing layer.conf using bitbake within layerindex-web
+#
+# Copyright (C) 2016 Wind River Systems
+# Author: Liam R. Howlett 
+#
+# Licensed under the MIT license, see COPYING.MIT for details
+#
+
+import sys
+import os
+import os.path
+import utils
+import tempfile
+import re
+
+class LayerConfParse:
+def __init__(self, enable_tracking=False, logger=None, bitbakepath=None, 
tinfoil=None):
+import settings
+self.logger = logger
+
+if not bitbakepath:
+fetchdir = settings.LAYER_FETCH_DIR
+bitbakepath = os.path.join(fetchdir, 'bitbake')
+self.bbpath = bitbakepath
+
+# Set up BBPATH.
+os.environ['BBPATH'] = str("%s" % self.bbpath)
+self.tinfoil = tinfoil
+
+if not self.tinfoil:
+self.tinfoil = utils.setup_tinfoil(self.bbpath, enable_tracking)
+
+self.config_data_copy = bb.data.createCopy(self.tinfoil.config_data)
+
+def parse_layer(self, layerbranch, layerdir):
+
+utils.checkout_layer_branch(layerbranch, layerdir, self.logger)
+
+
+# This is not a valid layer, parsing will cause exception.
+if not utils.is_layer_valid(layerdir):
+return None
+
+utils.parse_layer_conf(layerdir, self.config_data_copy, 
logger=self.logger)
+return self.config_data_copy
+
+def shutdown(self):
+self.tinfoil.shutdown()
+
+
diff --git a/layerindex/tools/import_layer.py b/layerindex/tools/import_layer.py
index 184c5cc..1daaeb2 100755
--- a/layerindex/tools/import_layer.py
+++ b/layerindex/tools/import_layer.py
@@ -19,6 +19,7 @@ import glob
 import utils
 import logging
 import subprocess
+from layerconfparse import LayerConfParse
 
 class DryRunRollbackException(Exception):
 pass
@@ -367,11 +368,21 @@ def main():
 if layer.name != settings.CORE_LAYER_NAME:
 if not core_layer:
 core_layer = utils.get_layer(settings.CORE_LAYER_NAME)
+
 if core_layer:
+logger.debug('Adding dep %s to %s' % (core_layer.name, 
layer.name))
 layerdep = LayerDependency()
 layerdep.layerbranch = layerbranch
 layerdep.dependency = core_layer
 layerdep.save()
+try:
+layerconfparser = LayerConfParse(logger=logger)
+config_data = layerconfparser.parse_layer(layerbranch, 
layerdir)
+finally:
+layerconfparser.shutdown()
+if config_data:
+utils.add_dependencies(layerbranch, config_data, 
logger=logger)
+
 
 # Get some extra meta-information
 readme_files = glob.glob(os.path.join(layerdir, 'README*'))
diff --git a/layerindex/update.py b/layerindex/update.py
index 423eb53..8684d36 100755
--- a/layerindex/update.py
+++ b/layerindex/update.py
@@ -16,6 +16,7 @@ import subprocess
 import signal
 from distutils.version import LooseVersion
 import utils
+from layerconfparse import LayerConfParse
 
 import warnings
 warnings.filterwarnings("ignore", category=DeprecationWarning)
@@ -92,7 +93,7 @@ def main():
 
 utils.setup_django()
 import settings
-from layerindex.models import Branch, LayerItem
+from layerindex.models import Branch, LayerItem, LayerDependency
 
 logger.setLevel(options.loglevel)
 
@@ -169,6 +170,7 @@ def main():
 # We now do this by calling out to a separate script; doing otherwise 
turned out to be
 # unreliable due to leaking memory (we're using bitbake internals in a 
manner in which
 # they never get used during normal operation).
+last_rev = {}
 for branch in branches:
 for layer in layerquery:
 if layer.vcs_url in failedrepos:
@@ -178,6 +180,10 @@ def main():
 repodir = os.path.join(fetchdir, urldir)

[yocto] [layerindex-web][PATCH v2 07/12] layerindex/recipeparse.py: refactor setup_tinfoil, checkout_layer_branch, parse_layer_conf to utils.py

2016-10-07 Thread Liam R. Howlett 
Move functions to utils to be used by other classes.
_
Signed-off-by: Liam R. Howlett 
---
 layerindex/bulkchange.py  |  4 ++--
 layerindex/recipeparse.py | 39 +++---
 layerindex/utils.py   | 48 +++
 3 files changed, 53 insertions(+), 38 deletions(-)

diff --git a/layerindex/bulkchange.py b/layerindex/bulkchange.py
index 8102571..ac9079f 100644
--- a/layerindex/bulkchange.py
+++ b/layerindex/bulkchange.py
@@ -42,9 +42,9 @@ def generate_patches(tinfoil, fetchdir, changeset, outputdir):
 patchname = "%s.patch" % layer.name
 patches.append(patchname)
 layerfetchdir = os.path.join(fetchdir, 
layer.get_fetch_dir())
-recipeparse.checkout_layer_branch(layerbranch, 
layerfetchdir)
+utils.checkout_layer_branch(layerbranch, layerfetchdir)
 layerdir = os.path.join(layerfetchdir, 
layerbranch.vcs_subdir)
-config_data_copy = 
recipeparse.setup_layer(tinfoil.config_data, fetchdir, layerdir, layer, 
layerbranch)
+config_data_copy = utils.setup_layer(tinfoil.config_data, 
fetchdir, layerdir, layer, layerbranch)
 if outfile:
 outfile.close()
 outfile = open(os.path.join(tmpoutdir, patchname), 'w')
diff --git a/layerindex/recipeparse.py b/layerindex/recipeparse.py
index a9cecfc..91a083b 100644
--- a/layerindex/recipeparse.py
+++ b/layerindex/recipeparse.py
@@ -20,31 +20,6 @@ class RecipeParseError(Exception):
 def __str__(self):
 return self.msg
 
-def _setup_tinfoil(bitbakepath, enable_tracking):
-sys.path.insert(0, bitbakepath + '/lib')
-import bb.tinfoil
-import bb.cooker
-import bb.data
-try:
-tinfoil = bb.tinfoil.Tinfoil(tracking=enable_tracking)
-except TypeError:
-# old API
-tinfoil = bb.tinfoil.Tinfoil()
-if enable_tracking:
-tinfoil.cooker.enableDataTracking()
-tinfoil.prepare(config_only = True)
-
-return tinfoil
-
-def _parse_layer_conf(layerdir, data):
-data.setVar('LAYERDIR', str(layerdir))
-if hasattr(bb, "cookerdata"):
-# Newer BitBake
-data = bb.cookerdata.parse_config_file(os.path.join(layerdir, "conf", 
"layer.conf"), data)
-else:
-# Older BitBake (1.18 and below)
-data = bb.cooker._parse(os.path.join(layerdir, "conf", "layer.conf"), 
data)
-data.expandVarref('LAYERDIR')
 
 
 def init_parser(settings, branch, bitbakepath, enable_tracking=False, 
nocheckout=False, classic=False, logger=None):
@@ -97,7 +72,7 @@ def init_parser(settings, branch, bitbakepath, 
enable_tracking=False, nocheckout
 tempdir = tempfile.mkdtemp(dir=settings.TEMP_BASE_DIR)
 os.chdir(tempdir)
 
-tinfoil = _setup_tinfoil(bitbakepath, enable_tracking)
+tinfoil = utils.setup_tinfoil(bitbakepath, enable_tracking)
 
 # Ensure TMPDIR exists (or insane.bbclass will blow up trying to write to 
the QA log)
 oe_tmpdir = tinfoil.config_data.getVar('TMPDIR', True)
@@ -110,14 +85,6 @@ def init_parser(settings, branch, bitbakepath, 
enable_tracking=False, nocheckout
 
 return (tinfoil, tempdir)
 
-def checkout_layer_branch(layerbranch, repodir, logger=None):
-if layerbranch.actual_branch:
-branchname = layerbranch.actual_branch
-else:
-branchname = layerbranch.branch.name
-out = utils.runcmd("git checkout origin/%s" % branchname, repodir, 
logger=logger)
-out = utils.runcmd("git clean -f -x", repodir, logger=logger)
-
 def setup_layer(config_data, fetchdir, layerdir, layer, layerbranch):
 # Parse layer.conf files for this layer and its dependencies
 # This is necessary not just because BBPATH needs to be set in order
@@ -125,7 +92,7 @@ def setup_layer(config_data, fetchdir, layerdir, layer, 
layerbranch):
 # or across layers, but also because custom variable values might be
 # set in layer.conf.
 config_data_copy = bb.data.createCopy(config_data)
-_parse_layer_conf(layerdir, config_data_copy)
+utils.parse_layer_conf(layerdir, config_data_copy)
 for dep in layerbranch.dependencies_set.all():
 depurldir = dep.dependency.get_fetch_dir()
 deprepodir = os.path.join(fetchdir, depurldir)
@@ -133,7 +100,7 @@ def setup_layer(config_data, fetchdir, layerdir, layer, 
layerbranch):
 if not deplayerbranch:
 raise RecipeParseError('Dependency %s of layer %s does not have 
branch record for branch %s' % (dep.dependency.name, layer.name, 
layerbranch.branch.name))
 deplayerdir = os.path.join(deprepodir, deplayerbranch.vcs_subdir)
-_parse_layer_conf(deplayerdir, config_data_copy)
+utils.parse_layer_conf(deplayerdir, config_data_copy)
 config_data_copy.delVar('LAYERDIR')
 return config_data_copy
 
diff --git a/layerindex/utils.py b/layerindex/utils.py
index 23b81f5..b634ce6 1006

[yocto] [layerindex-web][PATCH v2 06/12] layerindex/tools/import_project: Add import_project

2016-10-07 Thread Liam R. Howlett 
import_project will scan through a project and find any layer and add it
to the database by calling import_layer on each layer.  This differs
from import_layer as it tires to figure out the remote url and uses the
subdirectory (if one exists) as the name.

Signed-off-by: Liam R. Howlett 
---
 layerindex/tools/import_project.py | 180 +
 1 file changed, 180 insertions(+)
 create mode 100755 layerindex/tools/import_project.py

diff --git a/layerindex/tools/import_project.py 
b/layerindex/tools/import_project.py
new file mode 100755
index 000..511282d
--- /dev/null
+++ b/layerindex/tools/import_project.py
@@ -0,0 +1,180 @@
+#!/usr/bin/python3
+
+# Import a project into the database.
+#  This will scan through the directories in a project and find any layer and
+#  call import_layer.
+#
+#
+# Copyright (C) 2016 Wind River Systems
+# Author: Liam R. Howlett 
+#
+# Licensed under the MIT license, see COPYING.MIT for details
+
+from git import Repo
+from urllib.parse import urlparse
+import logging
+import optparse
+import os, fnmatch
+import sys
+
+sys.path.insert(0, os.path.realpath(os.path.join(os.path.dirname(__file__), 
'..')))
+
+import import_layer
+import update
+
+import utils
+
+
+
+
+class ImportProject:
+logger = utils.logger_create('ProjectIndexImport')
+
+def find_layers(self, path):
+self.logger.debug("finding layer..");
+result = []
+for root, dirs, files in os.walk(path, followlinks=True):
+for name in fnmatch.filter(files, 'layer.conf'):
+if not root.endswith('conf'):
+continue
+
+self.logger.debug("Found %s" % root)
+result.append(root)
+return result
+
+
+def main(self):
+parser = optparse.OptionParser(
+usage = """
+%prog [options] [directory]""")
+
+parser.add_option("-d", "--debug",
+help = "Enable debug output",
+action="store_const", const=logging.DEBUG, dest="loglevel", 
default=logging.INFO)
+parser.add_option("-n", "--dry-run",
+help = "Don't write any data back to the database",
+action="store_true", dest="dryrun")
+
+self.options, args = parser.parse_args(sys.argv)
+
+self.logger.setLevel(self.options.loglevel)
+
+if len(args) == 1:
+print("Please provide a directory.");
+sys.exit(1)
+
+install_dir = args[1]
+lc_list = self.find_layers(install_dir)
+core_layer = self.add_core(lc_list)
+if core_layer:
+lc_list.remove(core_layer)
+
+
+
+for layer in lc_list:
+self.add_layer(layer)
+
+def add_layer(self, layer):
+self.logger.debug("Processing layer %s" % layer);
+try:
+git_dir = utils.runcmd("git rev-parse --show-toplevel", 
destdir=layer, logger=self.logger)
+except Exception as e:
+self.logger.error("Cannot get root dir for layer %s: %s - 
Skipping." % (layer, str(e)))
+return 1
+
+repo = Repo(git_dir)
+actual_branch = repo.active_branch.name
+
+
+layer_name = layer.split('/')[-2]
+
+layer_subdir = None
+if os.path.basename(git_dir) != layer_name:
+layer_subdir = layer_name
+
+layer_name = self.get_layer_name(layer)
+
+for i in [1, 2, 3]:
+try:
+git_url = utils.runcmd("git config --get remote.origin.url", 
destdir=git_dir, logger=self.logger)
+except Exception as e:
+self.logger.info("Cannot get remote.origin.url for git dir %s: 
%s" % (git_dir, str(e)))
+
+if not os.path.exists(git_url):
+# Assume this is remote.
+self.logger.debug("Found git url = %s" % git_url)
+break;
+self.logger.debug("Iterating to find git url into %s" % git_dir)
+git_dir = git_url
+
+cmd = ['import_layer.py']
+if self.options.loglevel == logging.DEBUG:
+cmd.append("-d")
+if layer_subdir:
+cmd.append("-s")
+cmd.append(layer_subdir)
+
+if actual_branch:
+cmd.append("-a")
+cmd.append(actual_branch)
+cmd.append(git_url)
+cmd.append(layer_name)
+prefix = "Calling"
+
+if self.options.dryrun:
+prefix = "Would Call"
+
+
+self.logger.info("%s import_layer.main with %s for dir %s" % (prefix, 
str(cmd), layer))
+sys.argv = cmd
+if not self.options.dryrun:
+try:
+import_layer.main();
+except SystemExit as see:
+return see.code
+return 0
+
+def get_layer_name(self, layerconfdir):
+layer_name = layerconfdir.split('/')[-2]
+self.logger.debug('getting layer %s' % layerconfdir)
+layer_conf = os.path.join(layerconfdir, 'layer.conf')
+if os.pa

[yocto] [layerindex-web][PATCH v2 11/12] recipeparse: remove unnecessary else statement.

2016-10-07 Thread Liam R. Howlett 
Code clean up.

Signed-off-by: Liam R. Howlett 
---
 layerindex/recipeparse.py | 17 -
 1 file changed, 8 insertions(+), 9 deletions(-)

diff --git a/layerindex/recipeparse.py b/layerindex/recipeparse.py
index 91a083b..dd85bc3 100644
--- a/layerindex/recipeparse.py
+++ b/layerindex/recipeparse.py
@@ -134,15 +134,14 @@ def detect_file_type(path, subdir_start):
 if res:
 typename = 'machine'
 return (typename, None, res.group(1))
-else:
-res = bbclass_re.match(subpath)
-if res:
-typename = 'bbclass'
-return (typename, None, res.group(1))
-res = distro_conf_re.match(subpath)
-if res:
-typename = 'distro'
-return (typename, None, res.group(1))
+res = bbclass_re.match(subpath)
+if res:
+typename = 'bbclass'
+return (typename, None, res.group(1))
+res = distro_conf_re.match(subpath)
+if res:
+typename = 'distro'
+return (typename, None, res.group(1))
 
 if typename == 'recipe' or typename == 'bbappend':
 if subdir_start:
-- 
1.9.1

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [layerindex-web][PATCH v2 09/12] layerindex: Add collection and version to layerbranch

2016-10-07 Thread Liam R. Howlett 
Collection and version will be pulled from the layer.conf if it exists
and dependencies will be resolved by first checking for layers with the
dependency name and then checking for collections.  It is necessary to
shutdown tinfoil to avoid bitbake complaining about multiple instances.

Signed-off-by: Liam R. Howlett 
---
 layerindex/models.py   |  2 ++
 layerindex/update_layer.py |  1 +
 layerindex/utils.py| 20 ++--
 3 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/layerindex/models.py b/layerindex/models.py
index 2343ba7..cbfb244 100644
--- a/layerindex/models.py
+++ b/layerindex/models.py
@@ -130,6 +130,8 @@ class LayerItem(models.Model):
 class LayerBranch(models.Model):
 layer = models.ForeignKey(LayerItem)
 branch = models.ForeignKey(Branch)
+collection = models.CharField('Layer Collection', max_length=40, 
null=True, help_text='Name of the layer that could be used in the list of 
dependencies - can only contain letters, numbers and dashes')
+version = models.CharField('Layer Version', max_length=10, null=True, 
blank=True, help_text='The layer version for this particular branch.')
 vcs_subdir = models.CharField('Repository subdirectory', max_length=40, 
blank=True, help_text='Subdirectory within the repository where the layer is 
located, if not in the root (usually only used if the repository contains more 
than one layer)')
 vcs_last_fetch = models.DateTimeField('Last successful fetch', blank=True, 
null=True)
 vcs_last_rev = models.CharField('Last revision fetched', max_length=80, 
blank=True)
diff --git a/layerindex/update_layer.py b/layerindex/update_layer.py
index addeb4c..40d1f16 100644
--- a/layerindex/update_layer.py
+++ b/layerindex/update_layer.py
@@ -283,6 +283,7 @@ def main():
 layerconfparser.shutdown()
 sys.exit(1)
 utils.add_dependencies(layerbranch, layer_config_data, 
logger=logger)
+utils.set_layerbranch_collection_version(layerbranch, 
layer_config_data, logger=logger)
 layerbranch.save()
 
 layerrecipes = Recipe.objects.filter(layerbranch=layerbranch)
diff --git a/layerindex/utils.py b/layerindex/utils.py
index 018478e..170d202 100644
--- a/layerindex/utils.py
+++ b/layerindex/utils.py
@@ -30,8 +30,10 @@ def get_layer(layername):
 def get_dependency_layer(depname, version_str=None, logger=None):
 from layerindex.models import LayerItem, LayerBranch
 
-# Get any LayerBranch with a layer that has a name that matches the depname
-res = list(LayerBranch.objects.filter(layer__name=depname))
+# Get any LayerBranch with a layer that has a name that matches depmod, or
+# a LayerBranch that has the collection name depmod.
+res = list(LayerBranch.objects.filter(layer__name=depname)) + \
+  list(LayerBranch.objects.filter(collection=depname))
 
 # Nothing found, return.
 if not res:
@@ -68,6 +70,10 @@ def _add_dependency(var, name, layerbranch, config_data, 
logger=None):
 layer_name = layerbranch.layer.name
 var_name = layer_name
 
+if layerbranch.collection:
+var_name = layerbranch.collection
+
+
 dep_list = config_data.getVar("%s_%s" % (var, var_name), True)
 
 if not dep_list:
@@ -104,11 +110,21 @@ def _add_dependency(var, name, layerbranch, config_data, 
logger=None):
 
 if logger:
 logger.debug('Adding %s %s to %s' % (name, dep_layer.name, 
layer_name))
+
 layerdep = LayerDependency()
 layerdep.layerbranch = layerbranch
 layerdep.dependency = dep_layer
 layerdep.save()
 
+def set_layerbranch_collection_version(layerbranch, config_data, logger=None):
+
+layerbranch.collection = config_data.getVar('BBFILE_COLLECTIONS', 
True)
+ver_str = "LAYERVERSION_"
+if layerbranch.collection:
+layerbranch.collection = layerbranch.collection.strip()
+ver_str += layerbranch.collection
+layerbranch.version = config_data.getVar(ver_str, True)
+
 def setup_tinfoil(bitbakepath, enable_tracking):
 sys.path.insert(0, bitbakepath + '/lib')
 import bb.tinfoil
-- 
1.9.1

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [layerindex-web][PATCH v2 10/12] layerindexer: Add layer recommends support

2016-10-07 Thread Liam R. Howlett 
Parse layer.conf and add dependencies that are not required from
LAYERRECOMMENDS_.  Update the layerindex/template to support
recommends.  Uses bitbake parsing code & checks versions.

Signed-off-by: Liam R. Howlett 
---
 layerindex/models.py |  7 +++
 layerindex/tools/import_layer.py |  2 +-
 layerindex/update.py |  1 +
 layerindex/update_layer.py   |  1 +
 layerindex/utils.py  |  9 +++--
 templates/layerindex/detail.html | 32 +---
 6 files changed, 38 insertions(+), 14 deletions(-)

diff --git a/layerindex/models.py b/layerindex/models.py
index cbfb244..869d4a3 100644
--- a/layerindex/models.py
+++ b/layerindex/models.py
@@ -211,6 +211,12 @@ class LayerBranch(models.Model):
 return "%s: %s" % (self.layer.name, self.branch.name)
 
 
+def get_required(self):
+return self.dependencies_set.filter(required=True)
+
+def get_recommends(self):
+return self.dependencies_set.filter(required=False)
+
 class LayerMaintainer(models.Model):
 MAINTAINER_STATUS_CHOICES = (
 ('A', 'Active'),
@@ -232,6 +238,7 @@ class LayerMaintainer(models.Model):
 class LayerDependency(models.Model):
 layerbranch = models.ForeignKey(LayerBranch, 
related_name='dependencies_set')
 dependency = models.ForeignKey(LayerItem, related_name='dependents_set')
+required = models.BooleanField(default=True)
 
 class Meta:
 verbose_name_plural = "Layer dependencies"
diff --git a/layerindex/tools/import_layer.py b/layerindex/tools/import_layer.py
index 1daaeb2..377a0f0 100755
--- a/layerindex/tools/import_layer.py
+++ b/layerindex/tools/import_layer.py
@@ -382,7 +382,7 @@ def main():
 layerconfparser.shutdown()
 if config_data:
 utils.add_dependencies(layerbranch, config_data, 
logger=logger)
-
+utils.add_recommends(layerbranch, config_data, 
logger=logger)
 
 # Get some extra meta-information
 readme_files = glob.glob(os.path.join(layerdir, 'README*'))
diff --git a/layerindex/update.py b/layerindex/update.py
index 8684d36..6f93099 100755
--- a/layerindex/update.py
+++ b/layerindex/update.py
@@ -235,6 +235,7 @@ def main():
 continue
 
 utils.add_dependencies(layerbranch, config_data, 
logger=logger)
+utils.add_recommends(layerbranch, config_data, 
logger=logger)
 finally:
 layerconfparser.shutdown()
 
diff --git a/layerindex/update_layer.py b/layerindex/update_layer.py
index 40d1f16..27d4ebe 100644
--- a/layerindex/update_layer.py
+++ b/layerindex/update_layer.py
@@ -283,6 +283,7 @@ def main():
 layerconfparser.shutdown()
 sys.exit(1)
 utils.add_dependencies(layerbranch, layer_config_data, 
logger=logger)
+utils.add_recommends(layerbranch, layer_config_data, logger=logger)
 utils.set_layerbranch_collection_version(layerbranch, 
layer_config_data, logger=logger)
 layerbranch.save()
 
diff --git a/layerindex/utils.py b/layerindex/utils.py
index 170d202..484d3fe 100644
--- a/layerindex/utils.py
+++ b/layerindex/utils.py
@@ -62,9 +62,12 @@ def get_dependency_layer(depname, version_str=None, 
logger=None):
 return None
 
 def add_dependencies(layerbranch, config_data, logger=None):
-_add_dependency("LAYERDEPENDS", 'dependency', layerbranch, config_data, 
logger)
+_add_dependency("LAYERDEPENDS", 'dependency', layerbranch, config_data, 
logger=logger)
 
-def _add_dependency(var, name, layerbranch, config_data, logger=None):
+def add_recommends(layerbranch, config_data, logger=None):
+_add_dependency("LAYERRECOMMENDS", 'recommends', layerbranch, config_data, 
logger=logger, required=False)
+
+def _add_dependency(var, name, layerbranch, config_data, logger=None, 
required=True):
 from layerindex.models import LayerBranch, LayerDependency
 
 layer_name = layerbranch.layer.name
@@ -97,6 +100,7 @@ def _add_dependency(var, name, layerbranch, config_data, 
logger=None):
 logger.error('Error getting %s %s for %s\n%s' %(name, dep. 
layer_name, str(vse)))
 continue
 
+# No layer found.
 if not dep_layer:
 if logger:
 logger.error('Cannot resolve %s %s (version %s) for %s' % 
(name, dep, ver_str, layer_name))
@@ -114,6 +118,7 @@ def _add_dependency(var, name, layerbranch, config_data, 
logger=None):
 layerdep = LayerDependency()
 layerdep.layerbranch = layerbranch
 layerdep.dependency = dep_layer
+layerdep.required = required
 layerdep.save()
 
 def set_layerbranch_collection_version(layerbranch, config_data, logger=None):
diff --git a/templates/layerindex/detail.html b/templates/layerindex/detail.html
index 4bd7ecf..9d3ee05 100644
--- a/templates/layerindex/detail.html
+++ b/templates/layerindex/detai

[yocto] [layerindex-web][PATCH v2 04/12] layerindex/utils: Update runcmd to decode binary strings to strings.

2016-10-07 Thread Liam R. Howlett 
Convert binary strings to strings and strip leading/trailing whitespace
prior to returning errors and output.

Signed-off-by: Liam R. Howlett 
---
 layerindex/utils.py | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/layerindex/utils.py b/layerindex/utils.py
index 50268e0..23b81f5 100644
--- a/layerindex/utils.py
+++ b/layerindex/utils.py
@@ -32,7 +32,8 @@ def runcmd(cmd, destdir=None, printerr=True, logger=None):
 execute command, raise CalledProcessError if fail
 return output if succeed
 """
-#logger.debug("run cmd '%s' in %s" % (cmd, os.getcwd() if destdir is None 
else destdir))
+if logger:
+logger.debug("run cmd '%s' in %s" % (cmd, os.getcwd() if destdir is 
None else destdir))
 out = tempfile.TemporaryFile()
 try:
 subprocess.check_call(cmd, stdout=out, stderr=out, cwd=destdir, 
shell=True)
@@ -40,6 +41,7 @@ def runcmd(cmd, destdir=None, printerr=True, logger=None):
 out.seek(0)
 if printerr:
 output = out.read()
+output = output.decode('ascii').strip()
 if logger:
 logger.error("%s" % output)
 else:
@@ -49,7 +51,9 @@ def runcmd(cmd, destdir=None, printerr=True, logger=None):
 
 out.seek(0)
 output = out.read()
-#logger.debug("output: %s" % output.rstrip() )
+output = output.decode('ascii').strip()
+if logger:
+logger.debug("output: %s" % output.rstrip() )
 return output
 
 def setup_django():
-- 
1.9.1

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [layerindex-web][PATCH v2 12/12] layerindex/update_layer.py: Preserve the recipedependency files

2016-10-07 Thread Liam R. Howlett 
From: Mark Hatle 

In order to keep primary keys from constantly changing, preverse the
exist keys as much as possible.

Signed-off-by: Mark Hatle 
Signed-off-by: Liam R. Howlett 
---
 layerindex/update_layer.py | 19 ++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/layerindex/update_layer.py b/layerindex/update_layer.py
index 27d4ebe..ec40806 100644
--- a/layerindex/update_layer.py
+++ b/layerindex/update_layer.py
@@ -88,13 +88,30 @@ def update_recipe_file(tinfoil, data, path, recipe, 
layerdir_start, repodir):
 if depstr.startswith(layerdir_start) and not 
depstr.endswith('/conf/layer.conf'):
 filedeps.append(os.path.relpath(depstr, repodir))
 from layerindex.models import RecipeFileDependency
-RecipeFileDependency.objects.filter(recipe=recipe).delete()
+
+recipedeps_delete = []
+
+recipedeps = RecipeFileDependency.objects.filter(recipe=recipe)
+
+for values in recipedeps.values('path'):
+if 'path' in values:
+recipedeps_delete.append(values['path'])
+
 for filedep in filedeps:
+if filedep in recipedeps_delete:
+recipedeps_delete.remove(filedep)
+continue
+# New item, add it...
 recipedep = RecipeFileDependency()
 recipedep.layerbranch = recipe.layerbranch
 recipedep.recipe = recipe
 recipedep.path = filedep
 recipedep.save()
+
+for filedep in recipedeps_delete:
+logger.debug('%s: removing %s' % (recipe.layerbranch, filedep))
+recipedeps.filter(path=filedep).delete()
+
 except KeyboardInterrupt:
 raise
 except BaseException as e:
-- 
1.9.1

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [meta-security][PATCH] sleuthkit: fix No GNU_HASH in the elf binary

2016-10-07 Thread Armin Kuster 
From: Armin Kuster 

Signed-off-by: Armin Kuster 
---
 recipes-forensic/sleuth/sleuthkit_4.1.3.bb | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/recipes-forensic/sleuth/sleuthkit_4.1.3.bb 
b/recipes-forensic/sleuth/sleuthkit_4.1.3.bb
index 4b82616..4d40586 100644
--- a/recipes-forensic/sleuth/sleuthkit_4.1.3.bb
+++ b/recipes-forensic/sleuth/sleuthkit_4.1.3.bb
@@ -26,3 +26,5 @@ EXTRA_OECONF += "--enable-static=no --disable-java 
LIBS='-L${STAGING_LIBDIR}' LD
 FILES_${PN} += " ${datadir}/tsk"
 
 RDEPENDS_${PN} += " perl"
+
+INSANE_SKIP_${PN} = "ldflags"
-- 
2.3.5

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] [meta-security][PATCH] sleuthkit: fix No GNU_HASH in the elf binary

2016-10-07 Thread Khem Raj 

> On Oct 7, 2016, at 9:23 AM, Armin Kuster  wrote:
> 
> From: Armin Kuster 
> 
> Signed-off-by: Armin Kuster 
> ---
> recipes-forensic/sleuth/sleuthkit_4.1.3.bb | 2 ++
> 1 file changed, 2 insertions(+)
> 
> diff --git a/recipes-forensic/sleuth/sleuthkit_4.1.3.bb 
> b/recipes-forensic/sleuth/sleuthkit_4.1.3.bb
> index 4b82616..4d40586 100644
> --- a/recipes-forensic/sleuth/sleuthkit_4.1.3.bb
> +++ b/recipes-forensic/sleuth/sleuthkit_4.1.3.bb
> @@ -26,3 +26,5 @@ EXTRA_OECONF += "--enable-static=no --disable-java 
> LIBS='-L${STAGING_LIBDIR}' LD
> FILES_${PN} += " ${datadir}/tsk"
> 
> RDEPENDS_${PN} += " perl"
> +
> +INSANE_SKIP_${PN} = “ldflags"

Can we fix the makefile to respect linker flags instead ?

> --
> 2.3.5
> 
> --
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto



signature.asc
Description: Message signed with OpenPGP using GPGMail
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Y-AB PublishArtifacts toolchain wrong dir

2016-10-07 Thread gmane 

Hi,

I use the latest and greatest yocto autobuilder: 
f9b4e02730a5e712ee9085e2480f9a3b8b00f56d


It looks like Publishing Artifacts tries to find the toolchain instead 
of build/build/tmp/deploy/sdk (where it is built) here 
build/build/tmp/deploy/images/toolchain-x86_64


This patch [1] fixes the problem for me.

Regards,

Robert

https://github.com/RobertBerger/autobuilder-patches/blob/master/before-install/0004-PublichArtifacts-searches-toolchain-in-wrong-place.patch

--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] [layerindex-web][PATCH v2 00/12] Add Distro, dependency and recommends detection, and import_project

2016-10-07 Thread Mark Hatle 
FYI, I have made sure these are re-based on top of paule/django18 and pushed to:

git://git.yoctoproject.org/layerindex-web mhatle/django18

BTW for me upgrading from paule/django18 to this version.  I only had to run
'manage.py syncdb'.

This created the new database and added the missing field for the recommend.

I then ran update.py --forcereload and everything showed up as expected.

--Mark

On 10/7/16 10:57 AM, Liam R. Howlett wrote:
> This set of patches adds a number of features to the layerindex code:
>  -  Adds the ability to set the actual-branch to import_layer.
>  -  Adds distro to the database and web interface.
>  -  Adds collection & version information to layerbranch.
>  -  Uses layer name and collection name to determine dependencies and
> recommends.
>  -  Adds import_project to simplify pulling all layers and openembedded-core
> into a layer index.
> 
> Liam R. Howlett (11):
>   import_layer: Add --actual-branch option
>   layerindex/tools/import_layer.py: Sanitize layer name.
>   layerindex/tools/import_layer.py: Avoid failing if there is any layer
> to add.
>   layerindex/utils: Update runcmd to decode binary strings to strings.
>   layerindex: Add distro to web interface and model.
>   layerindex/tools/import_project: Add import_project
>   layerindex/recipeparse.py: refactor setup_tinfoil,
> checkout_layer_branch, parse_layer_conf to utils.py
>   layerindex: Detect dependencies from layer.conf files
>   layerindex: Add collection and version to layerbranch
>   layerindexer: Add layer recommends support
>   recipeparse: remove unnecessary else statement.
> 
> Mark Hatle (1):
>   layerindex/update_layer.py: Preserve the recipedependency files
> 
>  layerindex/admin.py|  11 +++
>  layerindex/bulkchange.py   |   4 +-
>  layerindex/layerconfparse.py   |  50 +++
>  layerindex/models.py   |  23 +
>  layerindex/recipeparse.py  |  53 +++
>  layerindex/restviews.py|  10 ++-
>  layerindex/tools/import_classic.py |   2 +
>  layerindex/tools/import_layer.py   |  47 +-
>  layerindex/tools/import_project.py | 180 
> +
>  layerindex/update.py   |  42 -
>  layerindex/update_layer.py |  87 --
>  layerindex/urls.py |   3 +
>  layerindex/urls_branch.py  |   6 +-
>  layerindex/utils.py| 159 +++-
>  layerindex/views.py|  29 +-
>  templates/layerindex/detail.html   |  55 +---
>  templates/layerindex/distros.html  |  76 
>  templates/layerindex/layers.html   |   1 +
>  templates/layerindex/machines.html |   1 +
>  templates/layerindex/recipes.html  |   1 +
>  20 files changed, 767 insertions(+), 73 deletions(-)
>  create mode 100644 layerindex/layerconfparse.py
>  create mode 100755 layerindex/tools/import_project.py
>  create mode 100644 templates/layerindex/distros.html
> 

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [[yocto-autobuilder][PATCHv4]] ] Build sets to test new OS distribution in autobuilder

2016-10-07 Thread Aníbal Limón 
From: Monserrat Sedeno 

As part of the process to set new OS distribution as supported on Yoctoc Project
a new patch was created with the list of build sets that should be executed.

Detailed information:
https://wiki.yoctoproject.org/wiki/Distro_Testing_Plan#Execute_Build_Sets

Fixes [YOCTO #9905]

Note:In order to add a new distro as supported all the build sets should be PASS

Signed-off-by: Monserrat Sedeno 
Signed-off-by: Aníbal Limón 
---
 .../nightly-qa-distro.conf | 55 ++
 1 file changed, 55 insertions(+)
 create mode 100644 buildset-config.autobuilder-qa/nightly-qa-distro.conf

diff --git a/buildset-config.autobuilder-qa/nightly-qa-distro.conf 
b/buildset-config.autobuilder-qa/nightly-qa-distro.conf
new file mode 100644
index 000..def4118
--- /dev/null
+++ b/buildset-config.autobuilder-qa/nightly-qa-distro.conf
@@ -0,0 +1,55 @@
+[nightly-qa-distro]
+builders: 'example-worker'
+repos: [{'poky':
+{'repourl':'git://git.yoctoproject.org/poky',
+ 'layerversion':{'core':'meta', 'yoctobsp':'meta-yocto-bsp', 
'yocto':'meta-yocto', 'poky':'meta-poky'},
+ 'branch':'master'}},
+{'oecore':
+{'repourl':'git://git.openembedded.org/openembedded-core',
+ 'layerversion':'core',
+ 'checkout':False,
+ 'branch':'master'}},
+{'bitbake':
+{'repourl':'git://git.openembedded.org/bitbake',
+ 'checkout':False,
+ 'branch':'master'}},
+{'eclipse-poky-neon':
+{'repourl':'git://git.yoctoproject.org/eclipse-poky',
+ 'checkout':False,
+ 'branch':'neon-master'}},
+{'meta-qt4':
+{'repourl':'git://git.yoctoproject.org/meta-qt4',
+ 'branch':'master'}},
+{'meta-qt3':
+{'repourl':'git://git.yoctoproject.org/meta-qt3',
+ 'branch':'master'}}]
+props: [{'fp_date':{'prop_type':'StringParameter',
+   'size': 20,
+   'name': 'fp_date',
+   'label':'Full Pass date:'}}]
+steps: [{'SetDest':{}},
+{'CheckOutLayers': {}},
+{'RunPreamble': {}},
+{'TriggerBuilds': {'schedulerName': 'dist-build', 
+ 'waitForFinish': 'True',
+ 'schedulerNames': {
+ 'nightly-oe-selftest': {},
+ 'nightly-oecore': {},
+ 'nightly-deb': {},
+ 'nightly-ipk': {},
+ 'nightly-world': {}, 
+ 'nightly-arm': {}, 
+ 'nightly-multilib': {}, 
+ 'nightly-x86-64-lsb': {},
+ 'nightly-x86': {},
+ 'nightly-x32': {},
+ 'poky-tiny': {}, 
+ 'nightly-qa-systemd': {},
+ 'nightly-qa-extras': {}, 
+ 'nightly-qa-targetbuilds': {},
+ 'nightly-qa-skeleton': {},
+ 'nightly-qa-logrotate': {}, 
+ 'nightly-qa-pam': {},
+ 'buildtools': {}},
+ 'eclipse-plugin-neon': {},
+ 'schedulerNames_nowait': {}}},]
-- 
2.1.4

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] openjdk build fails due to checksum mismatches from icedtea-native

2016-10-07 Thread Randy Mortensen 

> On Oct 6, 2016, at 8:35 PM, Khem Raj  wrote:
> 
>> 
>> On Oct 6, 2016, at 7:07 PM, Randy Mortensen > > wrote:
>> 
>> 
>>> On Oct 5, 2016, at 7:14 PM, Darcy Watkins  
>>> wrote:
>>> 
>>> 
>>> 
 On Oct 5, 2016, at 4:52 PM, Khem Raj  wrote:
> On Oct 5, 2016, at 4:45 PM, Randy Mortensen  
> wrote:
>> On Oct 5, 2016, at 5:04 PM, Darcy Watkins  
>> wrote:
>> 
>> From what I gleaned from recent discussions of fetcher errors, this is 
>> somehow connected with rollout of Python related security fixes to 
>> various Linux distributions and/or some ...-native recipes.
>> 
>> It was a bunch of tar balls that are named as mercurial hashes from 
>> within iced tea rather than the yocto fetch. I worked around it by 
>> grabbing the tarballs from a different checkout since I didn't have time 
>> to dig into it.
>> 
>> It affected a fresh checkout I was building from scratch.
>> 
> Thanks for the response. This also happened to me when trying to build 
> from scratch.
> For my clarification, did you already have the tar balls downloaded or 
> were you able to download them from a previous (icedtea) commit somehow?
>>> 
>>> I had the tar balls in a different build that I had around for some time.  
>>> The reason I never cached these ones in a shared location on our server was 
>>> I felt that tar balls with small hashes as filenames was too prone to 
>>> collisions, especially without a package name as a prefix.  I don't know if 
>>> that is a convention of iced tea, or how the fetcher handles mercurial.
>>> 
 Can you check if the tarballs have been rebuilt upstream ? if so we should 
 try to find out what changed.
 It could also be an oversight that a recipe update forgot or updated the 
 checksums wrongly. but we should try to root cause it
>>> 
>>> I agree here.  We should root cause it.
>>> 
>>> 
>> I’m not sure how this is all supposed to work, but I managed to get past the 
>> fetch failures by changing the md5sum and sha256sum checksums in 
>> icedtea7-native_2.1.3.bb.
>> I used the the checksums helpfully suggested by bitbake when it reported the 
>> errors.
>> 
>> I compared one of the problematic tar balls with a “good” one from a 
>> previous download and the only change I could identify was 3 extra lines 
>> added to a hidden file .hgtag  (which I presume maps a tag to a commit). Not 
>> sure why requesting the same hg commit results in a different tarball output.
>> 
> 
> could it be that its generating the tarballs from mercurial directly and 
> thats flawed somehow ?
That seems likely but I don’t know how to fix that.

I also don’t understand why the configure task fails after the fetch task 
successfully downloads the tarballs.
> 
> 
>> Now however iced tea fails to configure due to checksum errors. The 
>> configure task seems to re-download each tarball and check the sha256sum 
>> which is failing.
>> 
>> I’m not sure where to go from here to try and resolve so any more help is 
>> welcome.

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] openjdk build fails due to checksum mismatches from icedtea-native

2016-10-07 Thread Khem Raj 
On Fri, Oct 7, 2016 at 4:14 PM, Randy Mortensen  wrote:
>
> On Oct 6, 2016, at 8:35 PM, Khem Raj  wrote:
>
>
> On Oct 6, 2016, at 7:07 PM, Randy Mortensen  wrote:
>
>
> On Oct 5, 2016, at 7:14 PM, Darcy Watkins 
> wrote:
>
>
>
> On Oct 5, 2016, at 4:52 PM, Khem Raj  wrote:
>
> On Oct 5, 2016, at 4:45 PM, Randy Mortensen 
> wrote:
>
> On Oct 5, 2016, at 5:04 PM, Darcy Watkins 
> wrote:
>
> From what I gleaned from recent discussions of fetcher errors, this is
> somehow connected with rollout of Python related security fixes to various
> Linux distributions and/or some ...-native recipes.
>
> It was a bunch of tar balls that are named as mercurial hashes from within
> iced tea rather than the yocto fetch. I worked around it by grabbing the
> tarballs from a different checkout since I didn't have time to dig into it.
>
> It affected a fresh checkout I was building from scratch.
>
> Thanks for the response. This also happened to me when trying to build from
> scratch.
> For my clarification, did you already have the tar balls downloaded or were
> you able to download them from a previous (icedtea) commit somehow?
>
>
> I had the tar balls in a different build that I had around for some time.
> The reason I never cached these ones in a shared location on our server was
> I felt that tar balls with small hashes as filenames was too prone to
> collisions, especially without a package name as a prefix.  I don't know if
> that is a convention of iced tea, or how the fetcher handles mercurial.
>
> Can you check if the tarballs have been rebuilt upstream ? if so we should
> try to find out what changed.
> It could also be an oversight that a recipe update forgot or updated the
> checksums wrongly. but we should try to root cause it
>
>
> I agree here.  We should root cause it.
>
>
> I’m not sure how this is all supposed to work, but I managed to get past the
> fetch failures by changing the md5sum and sha256sum checksums in
> icedtea7-native_2.1.3.bb.
> I used the the checksums helpfully suggested by bitbake when it reported the
> errors.
>
> I compared one of the problematic tar balls with a “good” one from a
> previous download and the only change I could identify was 3 extra lines
> added to a hidden file .hgtag  (which I presume maps a tag to a commit). Not
> sure why requesting the same hg commit results in a different tarball
> output.
>
>
> could it be that its generating the tarballs from mercurial directly and
> thats flawed somehow ?
>
> That seems likely but I don’t know how to fix that.
>
> I also don’t understand why the configure task fails after the fetch task
> successfully downloads the tarballs.
>
>
>
> Now however iced tea fails to configure due to checksum errors. The
> configure task seems to re-download each tarball and check the sha256sum
> which is failing.
>

I wonder if this failure will still happen again when it rebuilds the
tarball internally. I have
a hunch that might happen again

> I’m not sure where to go from here to try and resolve so any more help is
> welcome.
>
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] [[yocto-autobuilder][PATCHv4]] ] Build sets to test new OS distribution in autobuilder

2016-10-07 Thread Bill Randle 
This version appears to ignore Beth's comments from yesterday where
she said buildset-config.autobuilder-qa wasn't really the appropriate
directory and suggested a different name, plus she said all buildable
configs should be present or links to them. If the later is true, then
not only should the patch use a different directory, but it should
also populate that new directory with links to the other build config
files referenced in this file.

-Bill

On Fri, Oct 7, 2016 at 1:50 PM, Aníbal Limón
 wrote:
> From: Monserrat Sedeno 
>
> As part of the process to set new OS distribution as supported on Yoctoc 
> Project
> a new patch was created with the list of build sets that should be executed.
>
> Detailed information:
> https://wiki.yoctoproject.org/wiki/Distro_Testing_Plan#Execute_Build_Sets
>
> Fixes [YOCTO #9905]
>
> Note:In order to add a new distro as supported all the build sets should be 
> PASS
>
> Signed-off-by: Monserrat Sedeno 
> Signed-off-by: Aníbal Limón 
> ---
>  .../nightly-qa-distro.conf | 55 
> ++
>  1 file changed, 55 insertions(+)
>  create mode 100644 buildset-config.autobuilder-qa/nightly-qa-distro.conf
>
> diff --git a/buildset-config.autobuilder-qa/nightly-qa-distro.conf 
> b/buildset-config.autobuilder-qa/nightly-qa-distro.conf
> new file mode 100644
> index 000..def4118
> --- /dev/null
> +++ b/buildset-config.autobuilder-qa/nightly-qa-distro.conf
> @@ -0,0 +1,55 @@
> +[nightly-qa-distro]
> +builders: 'example-worker'
> +repos: [{'poky':
> +{'repourl':'git://git.yoctoproject.org/poky',
> + 'layerversion':{'core':'meta', 'yoctobsp':'meta-yocto-bsp', 
> 'yocto':'meta-yocto', 'poky':'meta-poky'},
> + 'branch':'master'}},
> +{'oecore':
> +{'repourl':'git://git.openembedded.org/openembedded-core',
> + 'layerversion':'core',
> + 'checkout':False,
> + 'branch':'master'}},
> +{'bitbake':
> +{'repourl':'git://git.openembedded.org/bitbake',
> + 'checkout':False,
> + 'branch':'master'}},
> +{'eclipse-poky-neon':
> +{'repourl':'git://git.yoctoproject.org/eclipse-poky',
> + 'checkout':False,
> + 'branch':'neon-master'}},
> +{'meta-qt4':
> +{'repourl':'git://git.yoctoproject.org/meta-qt4',
> + 'branch':'master'}},
> +{'meta-qt3':
> +{'repourl':'git://git.yoctoproject.org/meta-qt3',
> + 'branch':'master'}}]
> +props: [{'fp_date':{'prop_type':'StringParameter',
> +   'size': 20,
> +   'name': 'fp_date',
> +   'label':'Full Pass date:'}}]
> +steps: [{'SetDest':{}},
> +{'CheckOutLayers': {}},
> +{'RunPreamble': {}},
> +{'TriggerBuilds': {'schedulerName': 'dist-build',
> + 'waitForFinish': 'True',
> + 'schedulerNames': {
> + 'nightly-oe-selftest': {},
> + 'nightly-oecore': {},
> + 'nightly-deb': {},
> + 'nightly-ipk': {},
> + 'nightly-world': {},
> + 'nightly-arm': {},
> + 'nightly-multilib': {},
> + 'nightly-x86-64-lsb': {},
> + 'nightly-x86': {},
> + 'nightly-x32': {},
> + 'poky-tiny': {},
> + 'nightly-qa-systemd': {},
> + 'nightly-qa-extras': {},
> + 'nightly-qa-targetbuilds': {},
> + 'nightly-qa-skeleton': {},
> + 'nightly-qa-logrotate': {},
> + 'nightly-qa-pam': {},
> + 'buildtools': {}},
> + 'eclipse-plugin-neon': {},
> + 'schedulerNames_nowait': {}}},]
> --
> 2.1.4
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Y-AB PublishArtifacts toolchain wrong dir

2016-10-07 Thread Bill Randle 
Hi Robert,

I see a couple of problems with this proposed patch. First, it's not
clear the existing code is broken. It calls getDeployNames() which
should detect the build artifact is "toolchain" and return a deploy
path of "deploy/sdk". If that is not working correctly, then that's
the code that should be fixed. Second, your patch dropped the "+
pipeline" from the command chain, which is used to set an error flag
in the event the copy failed. Third, you also dropped the portion of
the command chain that calculates the md5sums of the toolchain files.
(The fact that it appears, on the surface, that the md5sum files do
not get copied over would be a bug - although it seems like I tested
that at one time and it was working.)

-Bill

On Fri, Oct 7, 2016 at 8:44 AM,   wrote:
> Hi,
>
> I use the latest and greatest yocto autobuilder:
> f9b4e02730a5e712ee9085e2480f9a3b8b00f56d
>
> It looks like Publishing Artifacts tries to find the toolchain instead of
> build/build/tmp/deploy/sdk (where it is built) here
> build/build/tmp/deploy/images/toolchain-x86_64
>
> This patch [1] fixes the problem for me.
>
> Regards,
>
> Robert
>
> https://github.com/RobertBerger/autobuilder-patches/blob/master/before-install/0004-PublichArtifacts-searches-toolchain-in-wrong-place.patch
>
> --
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Y-AB PublishArtifacts toolchain wrong dir

2016-10-07 Thread gmane 

Hi,

On 2016-10-07 22:05, Bill Randle wrote:

Hi Robert,

I see a couple of problems with this proposed patch. First, it's not
clear the existing code is broken.


Here[1] is a paste of the error.

cp: cannot stat 
'/tmp/yocto-autobuilder/yocto-autobuilder/yocto-worker/res-custom-morty-multi-v7-core-image-minimal-dev-sato-sdk-toolchain/build/build/tmp/deploy/sdk/poky-*i686-core-image*.sh': 
No such file or directory


That's[2] the .conf file which I used.

Note that I'm not building a i686 toolchain, but only the 64 bit 
version.


[1] http://pastebin.com/aLyer2fi
[2] 
https://github.com/RobertBerger/meta-mainline/blob/morty-training-v4.4.x/multi-v7-ml/yocto-autobuilder/res-custom-morty-multi-v7-core-image-minimal-dev-sato-sdk-toolchain.conf


Regards,

Robert
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto