Re: [yocto] Pre-installing node.js packages
Hi, You can check this link: https://wiki.yoctoproject.org/wiki/TipsAndTricks/NPM I have recently added some fixes concerning npm. But I still have some issues using devtool: e.g the lockdown.json generation using the npm lockdown module is erroneous. I have no time for now, but I have planned to upstream some fixes. - Original Message - From: "Andy Pont" To: "Yocto-mailing-list" Sent: Tuesday, July 16, 2019 5:03:27 PM Subject: [yocto] Pre-installing node.js packages Hello, Is there a way to add node.js packages into a build? I’m trying to avoid having to include npm and compiler into my target image. -Andy. -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] [npm] duplicate code
Hi all, I am planning to do some work on npm and recipetool. I have seen that there is some duplicate code between bitbake/lib/bb/fetch2/npm.py and scripts/lib/recipetool/create_npm.py but there is some inconsistencies. Here is an example, the _parse_view() function which I patched recently has differ: https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/bitbake/lib/bb/fetch2/npm.py#n157 https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts/lib/recipetool/create_npm.py#n244 I think merging this "npm utility functions" is a good idea. But where ? I think about creating a new file: bitbake/lib/bb/npm_utils.py Or maybe use the existing bitbake/lib/bb/utils.py ? What do you think ? Jean-Marie LEMETAYER Enthusiast embedded systems engineer Savoir-faire Linux -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [npm] duplicate code
Hi Stefan, I have recently worked on a yocto project using npm and I have seen some issues. I have solved a few but only for bitbake: https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=eecba41822e86b69ebdb9cbc8fbfd512ad9a47d7 https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=a34d0d539e5fdf341541fb628652d22289e80512 https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=e90cd2ed61b93ee7e290e7e592f1f0242ab5c281 https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=a35abe31dc23916fd06afdb3a5e22a81ef3df579 As I have more time now, I wanted to continue my work by fixing devtool / recipetool. I have also checked the bugzilla for issues that I could fix / that should be tested again: https://bugzilla.yoctoproject.org/show_bug.cgi?id=10515 https://bugzilla.yoctoproject.org/show_bug.cgi?id=10760 https://bugzilla.yoctoproject.org/show_bug.cgi?id=11028 https://bugzilla.yoctoproject.org/show_bug.cgi?id=11029 https://bugzilla.yoctoproject.org/show_bug.cgi?id=12534 https://bugzilla.yoctoproject.org/show_bug.cgi?id=13415 I ended up with this todo list: - merge the duplicate code between bitbake and recipetool - fix the npm package name handling in recipetool - fix the npm package version handling in recipetool - fix the lockdown.json file generation in recipetool - create an example nodejs application to test all these cases - update the wiki using this example application: https://wiki.yoctoproject.org/wiki/TipsAndTricks/NPM Finally, in my recent project, we were using angular (https://angular.io) for the front-end. I have planned to add the work done to support angular applications in yocto (angular.bbclass) and update recipetool to handle them. Your work sounds very interesting. The good point is that npm-shrinkwrap.json and lockdown.json files (which have generation issues btw) will no longer be required. But projects using npm can have a lot of dependencies (e.g. the angular example app have 1053 dependencies). Is recipetool will be handling the whole recipes creation in one time ? Is it possible to see your work ? A public fork would be nice. I would gladly help you / test your work / add my fixes if needed. Best regards, Jean-Marie Lemetayer - Mail original - De: "Stefan Herbrechtsmeier" À: "Yocto-mailing-list" Envoyé: Vendredi 4 Octobre 2019 08:53:10 Objet: Re: [yocto] [npm] duplicate code Hi Jean-Marie, what work do you plan to do on npm and recipetool? I have start to rework the complete npm integration to synchronize it with other languages like python. I replace the bitbake npm fetch with the wget fetcher and download the npm tgz files direct. The idea is to handle every package as a separate recipe and move the package dependencies into the recipes. Therefore I parse the package.json and the npm server json to create a recipe. Regards Stefan Am 03.10.19 um 17:37 schrieb Jean-marie Lemetayer: > Hi all, > > I am planning to do some work on npm and recipetool. > > I have seen that there is some duplicate code between > bitbake/lib/bb/fetch2/npm.py and scripts/lib/recipetool/create_npm.py but > there is some inconsistencies. > > Here is an example, the _parse_view() function which I patched recently has > differ: > https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/bitbake/lib/bb/fetch2/npm.py#n157 > https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts/lib/recipetool/create_npm.py#n244 > > I think merging this "npm utility functions" is a good idea. But where ? > > I think about creating a new file: bitbake/lib/bb/npm_utils.py > > Or maybe use the existing bitbake/lib/bb/utils.py ? > > What do you think ? > > > Jean-Marie LEMETAYER > Enthusiast embedded systems engineer > Savoir-faire Linux > -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [npm] duplicate code
Hi Stefan, I thought about your idea of using Yocto to manage NPM package dependencies and I ran into an issue. NPM projects can have multiple dependencies on a single package, and sometimes with multiple versions. NPM will manage this by creating sub 'node_modules' tree. Here is an example with a newly created angular application. The app depends on 3 different version of 'ansi-regex'. NPM will install the packages this way: node_modules/ansi-regex @ 2.1.1 node_modules/cliui/node_modules/ansi-regex @ 3.0.0 node_modules/inquirer/node_modules/ansi-regex @ 4.1.0 node_modules/string-width/node_modules/ansi-regex @ 3.0.0 node_modules/@angular/compiler-cli/node_modules/ansi-regex @ 4.1.0 How can you handle that with Yocto ? I am not sure but I think it is not possible. For that reason I think I will base my developments on the master branch. - Mail original - > De: "Stefan Herbrechtsmeier" > À: "Jean-marie Lemetayer" > Cc: "Yocto-mailing-list" , "Savoir-faire Linux > Rennes" > Envoyé: Vendredi 4 Octobre 2019 16:55:47 > Objet: Re: [yocto] [npm] duplicate code > Hi Jean-Marie, > > Am 04.10.19 um 14:37 schrieb Jean-marie Lemetayer: >> > I have recently worked on a yocto project using npm and I have seen some >> > issues. >> > I have solved a few but only for bitbake: >> https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=eecba41822e86b69ebdb9cbc8fbfd512ad9a47d7 >> https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=a34d0d539e5fdf341541fb628652d22289e80512 >> https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=e90cd2ed61b93ee7e290e7e592f1f0242ab5c281 >> https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=a35abe31dc23916fd06afdb3a5e22a81ef3df579 >> >> As I have more time now, I wanted to continue my work by fixing devtool / >> recipetool. >> >> I have also checked the bugzilla for issues that I could fix / that should be >> tested again: >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=10515 >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=10760 >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=11028 >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=11029 >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12534 >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13415 >> >> I ended up with this todo list: >> - merge the duplicate code between bitbake and recipetool >> - fix the npm package name handling in recipetool >> - fix the npm package version handling in recipetool >> - fix the lockdown.json file generation in recipetool >> - create an example nodejs application to test all these cases >> - update the wiki using this example application: >> https://wiki.yoctoproject.org/wiki/TipsAndTricks/NPM >> >> Finally, in my recent project, we were using angular (https://angular.io) for >> the front-end. I have planned to add the work done to support angular >> applications in yocto (angular.bbclass) and update recipetool to handle them. > > Do you want to build an angular application? In this case you have to > produce a machine (independent) package via a native npm build because > you need to run the angular compiler on the build machine. Yes, I have an 'angular-cli_x.y.z.bb' and an 'angular.bbclass' which depends on 'angular-cli-native'. I have planned to upstream this work. > >> >> Your work sounds very interesting. The good point is that npm-shrinkwrap.json >> and lockdown.json files (which have generation issues btw) will no longer be >> required. But projects using npm can have a lot of dependencies (e.g. the >> angular example app have 1053 dependencies). > > A lot of dependencies are the problem at the moment. But many > dependencies are not update frequently and my current solution assume > that I can always use the latest version per major release version > >> Is recipetool will be handling the whole recipes creation in one time ? > > That's the problem at the moment. I don't manage to build multiple > recipes from one recipetool run. Therefore I have a two stage approach. > I use recipetool to build exact one recipe. A second script generates a > list of all dependencies and call recipetool per package if the recipe > doesn't already exist. This works but the run time is very high. > >> Is it possible to see your work ? A public fork would be nice. I would gladly >> help you / test your work / add my fixes if needed. > > I only have a unfinished Proof of Concept. If you like I could clean it > up a bit and post it on github as a separate layer. Like I said at the start of this email, I will not base my dev on yours. But I am still interested by seeing your work. Best regards, Jean-Marie -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] nodejs npm qemux86_64
Hi Jonas,
I am currently refactoring / updating the npm handling in Yocto:
http://lists.openembedded.org/pipermail/openembedded-core/2019-October/288352.html
http://lists.openembedded.org/pipermail/bitbake-devel/2019-October/020449.html
Your point is very interesting, I will take it into account for the next
patchset version.
Best regards,
Jean-Marie
- On Nov 7, 2019, at 10:56 PM, Jonas Andersson jonaskganders...@gmail.com
wrote:
> Hi,
>
> I have installation problems for npm recipes.
> Using Yocto thud and machine=qemux86_64(wrlinux-x), but as I can see it this
> concerns all
> use of npm.bbclass for 64-bits system.
>
> The problem seems to be the use of ${libdir} in npm.bbclass, when 64 bit is
> used
> this expands to lib64.
> npm install --prefix ${D}${prefix} installs to /usr/lib/node_modules which is
> not
> in sync with the following mv ${D}${libdir}/node_modules ${D}${libdir}/node.
>
> I made an quick fix by replacing ${libdir} by ${prefix}/lib/, but as I see it
> nodejs packages never is installed to lib64 in any other system as I can find
> information about.
>
> So the question is if the use of ${libdir} is bad and the path should always
> be
> to /lib/?
>
> Best regards
> Jonas
>
>
> --
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] nodejs npm qemux86_64
Actually I just update my patchset to use ${nonarch_libdir} instead of
${libdir}.
This will be on the next patchset version.
For your information you can check the meta/conf/bitbake.conf file for others
target filesystem paths.
Best regards,
Jean-Marie
On Nov 8, 2019, at 10:46 AM, Jean-Marie LEMETAYER
jean-marie.lemeta...@savoirfairelinux.com wrote:
> Hi Jonas,
>
> I am currently refactoring / updating the npm handling in Yocto:
> http://lists.openembedded.org/pipermail/openembedded-core/2019-October/288352.html
> http://lists.openembedded.org/pipermail/bitbake-devel/2019-October/020449.html
>
> Your point is very interesting, I will take it into account for the next
> patchset version.
>
> Best regards,
> Jean-Marie
>
> On Nov 7, 2019, at 10:56 PM, Jonas Andersson jonaskganders...@gmail.com wrote:
>> Hi,
>>
>> I have installation problems for npm recipes.
>> Using Yocto thud and machine=qemux86_64(wrlinux-x), but as I can see it this
>> concerns all
>> use of npm.bbclass for 64-bits system.
>>
>> The problem seems to be the use of ${libdir} in npm.bbclass, when 64 bit is
>> used
>> this expands to lib64.
>> npm install --prefix ${D}${prefix} installs to /usr/lib/node_modules which is
>> not
>> in sync with the following mv ${D}${libdir}/node_modules ${D}${libdir}/node.
>>
>> I made an quick fix by replacing ${libdir} by ${prefix}/lib/, but as I see it
>> nodejs packages never is installed to lib64 in any other system as I can find
>> information about.
>>
>> So the question is if the use of ${libdir} is bad and the path should always
>> be
>> to /lib/?
>>
>> Best regards
>> Jonas
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
