[yocto] [yocto-autobuilder][PATCH] bin/buildlogger: add new script to aid SWAT process
buildlogger will be started with the autobuilder and, when correctly
configured, monitor the AB's JSON API for newly started builds. When one is
detected information about the build will be posted to the wiki.
Requires a ConfigParser (ini) style configuration file at
AB_BASE/etc/buildlogger.conf formatted as follows:
[wikiuser]
username = botuser
password = botuserpassword
[wiki]
pagetitle = BuildLog
Signed-off-by: Joshua Lock
---
.gitignore | 2 +
bin/buildlogger | 273
yocto-start-autobuilder | 8 ++
yocto-stop-autobuilder | 45
4 files changed, 309 insertions(+), 19 deletions(-)
create mode 100755 bin/buildlogger
diff --git a/.gitignore b/.gitignore
index 3f9505b..48c8a85 100644
--- a/.gitignore
+++ b/.gitignore
@@ -8,6 +8,7 @@
###
buildset-config
config/autobuilder.conf
+etc/buildlogger.conf
# Everything else #
###
@@ -25,6 +26,7 @@ yocto-controller/controller.cfg
yocto-controller/state.sqlite
yocto-controller/twistd.log*
yocto-controller/buildbot.tac
+yocto-controller/logger.log
yocto-worker/build-appliance/build(newcommits)
yocto-worker/buildbot.tac
yocto-worker/janitor.log
diff --git a/bin/buildlogger b/bin/buildlogger
new file mode 100755
index 000..7b39f92
--- /dev/null
+++ b/bin/buildlogger
@@ -0,0 +1,273 @@
+#!/usr/bin/env python3
+'''
+Created on May 5, 2016
+
+__author__ = "Joshua Lock"
+__copyright__ = "Copyright 2016, Intel Corporation"
+__credits__ = ["Joshua Lock"]
+__license__ = "GPL"
+__version__ = "2.0"
+__maintainer__ = "Joshua Lock"
+__email__ = "joshua.g.l...@intel.com"
+'''
+
+# We'd probably benefit from using some caching, but first we'd need the AB API
+# to include
+#
+# We can set repo url, branch & commit for a bunch of repositorys.
+# Do they all get built for nightly?
+
+try:
+import configparser
+except ImportError:
+import ConfigParser as configparser
+import json
+import os
+import requests
+import signal
+import sys
+import time
+
+abapi =
"https://autobuilder.yoctoproject.org/main/json/builders/nightly/builds/_all";
+# Wiki editing params
+un = ''
+pw = ''
+wikiapi = "https://wiki.yoctoproject.org/wiki/api.php";
+title = ''
+
+last_logged = ''
+# TODO: probably shouldn't write files in the same location as the script?
+cachefile = 'buildlogger.lastbuild'
+tmpfile = '/tmp/.buildlogger.pid'
+
+
+# Load configuration information from an ini
+def load_config(configfile):
+global un
+global pw
+global title
+success = False
+
+if os.path.exists(configfile):
+try:
+config = configparser.ConfigParser()
+config.read(configfile)
+un = config.get('wikiuser', 'username')
+pw = config.get('wikiuser', 'password')
+title = config.get('wiki', 'pagetitle')
+success = True
+except configparser.Error as ex:
+print('Failed to load buildlogger configuration with error: %s' %
str(ex))
+else:
+print('Config file %s does not exist, please create and populate it.'
% configfile)
+
+return success
+
+# we can't rely on the built in JSON parser in the requests module because
+# the JSON we get from the wiki begins with a UTF-8 BOM which chokes
+# json.loads().
+# Thus we decode the raw resonse content into a string and load that into a
+# JSON object ourselves.
+#
+# http://en.wikipedia.org/wiki/Byte_Order_Mark
+# http://bugs.python.org/issue18958
+def parse_json(response):
+text = response.content.decode('utf-8-sig')
+
+return json.loads(text)
+
+
+# Get the current content of the BuildLog page -- to make the wiki page as
+# useful as possible the most recent log entry should be at the top, to
+# that end we need to edit the whole page so that we can insert the new entry
+# after the log but before the other entries.
+# This method fetches the current page content, splits out the blurb and
+# returns a pair:
+# 1) the blurb
+# 2) the current entries
+def wiki_get_content():
+params = '?format=json&action=query&prop=revisions&rvprop=content&titles='
+req = requests.get(wikiapi+params+title)
+parsed = parse_json(req)
+pageid = sorted(parsed['query']['pages'].keys())[-1]
+content = parsed['query']['pages'][pageid]['revisions'][0]['*']
+blurb, entries = content.split('==', 1)
+# ensure we keep only a single newline after the blurb
+blurb = blurb.strip() + "\n"
+entries = '=='+entries
+
+return blurb, entries
+
+
+# Login to the wiki and return cookies for the logged in session
+def wiki_login():
+payload = {
+'action': 'login',
+'lgname': un,
+'lgpassword': pw,
+'utf8': '',
+'format': 'json'
+}
+req1 = requests.post(wikiapi, data=payload)
+parsed = parse_json(req1)
+login_token = parsed['login']['token']
+
+payload['lgtoken'] = login_token
+req2 = requests.post(wikiapi,
Re: [yocto] HG Parameters for the fetcher
Thanks,
What is when the HG comes over SSH with authentication? Like:
SRC_URI = "hg://myHG/ABC;module="dev";protocol=ssh;rev=42"
with user for example "yocto"
I think the syntax could be: SRC_URI =
"hg://yocto@myHG/ABC;module="dev";protocol=ssh;rev=42"
Regards
Stefan Jaritz
Entwickler
ESA Elektroschaltanlagen Grimma GmbH
Broner Ring 30
04668 Grimma
Telefon: +49 3437 9211 176
Telefax: +49 3437 9211 26
E-Mail: s.jar...@esa-grimma.de
Internet: www.esa-grimma.de
Geschäftsführer:
Dipl.-Ing. Jörg Gaitzsch
Jörg Reinker
Sitz der Gesellschaft: Grimma
Ust.-ID: DE 141784437
Amtsgericht: Leipzig, HRB 5159
Steuernummer: 238/108/00755
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte
Informationen.
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich
erhalten
haben, informieren Sie bitte sofort den Absender und löschen Sie diese
Nachricht. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser
Mail
ist nicht gestattet.
This e-mail may contain confidential and/or privileged information. If you
are
not the intended recipient (or have received this e-mail in error) please
notify the sender immediately and destroy this e-mail. Any unauthorized
copying, disclosure or distribution of the material in this e-mail is
strictly
forbidden.
Von:"Jeyachandran, Yukatharsani (Contractor)"
An: "s.jar...@esa-grimma.de" , yocto
Datum: 02.05.2016 16:50
Betreff:Re: [yocto] HG Parameters for the fetcher
Hi,
Add your source file in this below format:-
For Example:-
SRCREV = "${AUTOREV}"
SRC_URI = "svn or git ://;module=" ";protocol=https or ssh"
Thanks and Regards,
Yukatharsani J.
From: yocto-boun...@yoctoproject.org on
behalf of s.jar...@esa-grimma.de
Sent: Monday, May 2, 2016 9:25 AM
To: yocto
Subject: [yocto] HG Parameters for the fetcher
Hej,
Actually I try to build a recipe for a Mercurial SCM. The hg sources are
local. In my bb File I have this lines:
SRCREV = "486c7001d4e5"
SRC_URI = "hg://myHG"
I got the error:
ERROR: Logfile of failure stored in:
/home/user/myTC/poky/build/tmp/work/cortexa5hf-vfp-poky-linux-gnueabi/myHG/0.1-r0/temp/log.do_fetch.27908
Log data follows:
| DEBUG: Executing python function do_fetch
| DEBUG: Executing python function base_do_fetch
| DEBUG: Python function base_do_fetch finished
| DEBUG: Python function do_fetch finished
| ERROR: Function failed: URL: 'hg://myHG' is missing the required
parameter 'module'
ERROR: Task 4
(/home/user/myTC/poky/meta-sek4/recipes-test/deamons/myHG_0.1.bb,
do_fetch) failed with exit code '1'
Any ideas?
Stefan Jaritz
Entwickler
ESA Elektroschaltanlagen Grimma GmbH
Broner Ring 30
04668 Grimma
Telefon: +49 3437 9211 176
Telefax: +49 3437 9211 26
E-Mail: s.jar...@esa-grimma.de
Internet: www.esa-grimma.de
Geschäftsführer:
Dipl.-Ing. Jörg Gaitzsch
Jörg Reinker
Sitz der Gesellschaft: Grimma
Ust.-ID: DE 141784437
Amtsgericht: Leipzig, HRB 5159
Steuernummer: 238/108/00755
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte
Informationen.
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich
erhalten
haben, informieren Sie bitte sofort den Absender und löschen Sie diese
Nachricht. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser
Mail
ist nicht gestattet.
This e-mail may contain confidential and/or privileged information. If you
are
not the intended recipient (or have received this e-mail in error) please
notify the sender immediately and destroy this e-mail. Any unauthorized
copying, disclosure or distribution of the material in this e-mail is
strictly
forbidden.
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
[yocto] Formulating Recipe for out-of-tree kernel module
I am looking to produce a recipe for the amfeltec usb-fxs adaptor
(http://amfeltec.com/products/piranha-usb-fxs-adapter/), with the intention of
contributing this to the meta-telephony layer.
However, I have found a few obstacles to getting a clean recipe:The makefile
produces both a kernel module and a user utility, with the latter being
hard-coded to build with g++. I can produce a recipe to produce one or other
of the components but not both. Given the structure of the build directories,
should this be be achievable, or should I be spitting up the recipe in
two?Dahdi header files and Modules.symvers are required. A header file is used
to deduce the version of Dahdi, Modules.symvers is required for the module
build process. Hence, requires the location of the Dahdi source/build
directory. This can be set relative to the amfeltec work directory but this
feels wrong.
In more general terms my questions are:Should I split a recipe into kernel and
non-kernel components?How should one out-of-tree recipe access the
headers/Module.symvers from another (should that recipe be installing them in
${STAGING_KERNEL_DIR} (or somewhere else)?)
Cheers,Chris
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [yocto-autobuilder][PATCH] bin/buildlogger: add new script to aid SWAT process
A few things.
On 9 May 2016 at 12:06, Joshua Lock wrote:
> buildlogger will be started with the autobuilder and, when correctly
> configured, monitor the AB's JSON API for newly started builds. When one is
> detected information about the build will be posted to the wiki.
>
> Requires a ConfigParser (ini) style configuration file at
> AB_BASE/etc/buildlogger.conf formatted as follows:
Can we get a buildlogger.conf.example in AB_BASE/etc?
>
> [wikiuser]
> username = botuser
> password = botuserpassword
>
> [wiki]
> pagetitle = BuildLog
>
> Signed-off-by: Joshua Lock
> ---
> .gitignore | 2 +
> bin/buildlogger | 273
>
> yocto-start-autobuilder | 8 ++
> yocto-stop-autobuilder | 45
> 4 files changed, 309 insertions(+), 19 deletions(-)
> create mode 100755 bin/buildlogger
>
> diff --git a/.gitignore b/.gitignore
> index 3f9505b..48c8a85 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -8,6 +8,7 @@
> ###
> buildset-config
> config/autobuilder.conf
> +etc/buildlogger.conf
>
> # Everything else #
> ###
> @@ -25,6 +26,7 @@ yocto-controller/controller.cfg
> yocto-controller/state.sqlite
> yocto-controller/twistd.log*
> yocto-controller/buildbot.tac
> +yocto-controller/logger.log
> yocto-worker/build-appliance/build(newcommits)
> yocto-worker/buildbot.tac
> yocto-worker/janitor.log
> diff --git a/bin/buildlogger b/bin/buildlogger
> new file mode 100755
> index 000..7b39f92
> --- /dev/null
> +++ b/bin/buildlogger
> @@ -0,0 +1,273 @@
> +#!/usr/bin/env python3
> +'''
> +Created on May 5, 2016
> +
> +__author__ = "Joshua Lock"
> +__copyright__ = "Copyright 2016, Intel Corporation"
> +__credits__ = ["Joshua Lock"]
> +__license__ = "GPL"
> +__version__ = "2.0"
> +__maintainer__ = "Joshua Lock"
> +__email__ = "joshua.g.l...@intel.com"
> +'''
> +
> +# We'd probably benefit from using some caching, but first we'd need the AB
> API
> +# to include
> +#
> +# We can set repo url, branch & commit for a bunch of repositorys.
> +# Do they all get built for nightly?
> +
> +try:
> +import configparser
> +except ImportError:
> +import ConfigParser as configparser
> +import json
> +import os
> +import requests
> +import signal
> +import sys
> +import time
> +
> +abapi =
> "https://autobuilder.yoctoproject.org/main/json/builders/nightly/builds/_all";
> +# Wiki editing params
> +un = ''
> +pw = ''
> +wikiapi = "https://wiki.yoctoproject.org/wiki/api.php";
> +title = ''
> +
> +last_logged = ''
> +# TODO: probably shouldn't write files in the same location as the script?
> +cachefile = 'buildlogger.lastbuild'
> +tmpfile = '/tmp/.buildlogger.pid'
> +
> +
> +# Load configuration information from an ini
> +def load_config(configfile):
> +global un
> +global pw
> +global title
> +success = False
> +
> +if os.path.exists(configfile):
> +try:
> +config = configparser.ConfigParser()
> +config.read(configfile)
> +un = config.get('wikiuser', 'username')
> +pw = config.get('wikiuser', 'password')
> +title = config.get('wiki', 'pagetitle')
> +success = True
> +except configparser.Error as ex:
> +print('Failed to load buildlogger configuration with error: %s'
> % str(ex))
> +else:
> +print('Config file %s does not exist, please create and populate
> it.' % configfile)
> +
> +return success
> +
> +# we can't rely on the built in JSON parser in the requests module because
> +# the JSON we get from the wiki begins with a UTF-8 BOM which chokes
> +# json.loads().
> +# Thus we decode the raw resonse content into a string and load that into a
> +# JSON object ourselves.
> +#
> +# http://en.wikipedia.org/wiki/Byte_Order_Mark
> +# http://bugs.python.org/issue18958
> +def parse_json(response):
> +text = response.content.decode('utf-8-sig')
> +
> +return json.loads(text)
> +
> +
> +# Get the current content of the BuildLog page -- to make the wiki page as
> +# useful as possible the most recent log entry should be at the top, to
> +# that end we need to edit the whole page so that we can insert the new entry
> +# after the log but before the other entries.
> +# This method fetches the current page content, splits out the blurb and
> +# returns a pair:
> +# 1) the blurb
> +# 2) the current entries
> +def wiki_get_content():
> +params =
> '?format=json&action=query&prop=revisions&rvprop=content&titles='
> +req = requests.get(wikiapi+params+title)
> +parsed = parse_json(req)
> +pageid = sorted(parsed['query']['pages'].keys())[-1]
> +content = parsed['query']['pages'][pageid]['revisions'][0]['*']
> +blurb, entries = content.split('==', 1)
> +# ensure we keep only a single newline after the blurb
> +blurb = blurb.strip() + "\n"
> +entries = '=='+entries
> +
> +return blurb, entries
> +
> +
> +# Login
Re: [yocto] [yocto-autobuilder][PATCH] bin/buildlogger: add new script to aid SWAT process
On Mon, 2016-05-09 at 12:42 +0100, Flanagan, Elizabeth wrote:
> A few things.
>
> On 9 May 2016 at 12:06, Joshua Lock wrote:
> >
> > buildlogger will be started with the autobuilder and, when
> > correctly
> > configured, monitor the AB's JSON API for newly started builds.
> > When one is
> > detected information about the build will be posted to the wiki.
> >
> > Requires a ConfigParser (ini) style configuration file at
> > AB_BASE/etc/buildlogger.conf formatted as follows:
> Can we get a buildlogger.conf.example in AB_BASE/etc?
Sure, I'll add that in a v2.
> >
> >
> > [wikiuser]
> > username = botuser
> > password = botuserpassword
> >
> > [wiki]
> > pagetitle = BuildLog
> >
> > Signed-off-by: Joshua Lock
> > ---
> > .gitignore | 2 +
> > bin/buildlogger | 273
> >
> > yocto-start-autobuilder | 8 ++
> > yocto-stop-autobuilder | 45
> > 4 files changed, 309 insertions(+), 19 deletions(-)
> > create mode 100755 bin/buildlogger
> >
> > diff --git a/.gitignore b/.gitignore
> > index 3f9505b..48c8a85 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -8,6 +8,7 @@
> > ###
> > buildset-config
> > config/autobuilder.conf
> > +etc/buildlogger.conf
> >
> > # Everything else #
> > ###
> > @@ -25,6 +26,7 @@ yocto-controller/controller.cfg
> > yocto-controller/state.sqlite
> > yocto-controller/twistd.log*
> > yocto-controller/buildbot.tac
> > +yocto-controller/logger.log
> > yocto-worker/build-appliance/build(newcommits)
> > yocto-worker/buildbot.tac
> > yocto-worker/janitor.log
> > diff --git a/bin/buildlogger b/bin/buildlogger
> > new file mode 100755
> > index 000..7b39f92
> > --- /dev/null
> > +++ b/bin/buildlogger
> > @@ -0,0 +1,273 @@
> > +#!/usr/bin/env python3
> > +'''
> > +Created on May 5, 2016
> > +
> > +__author__ = "Joshua Lock"
> > +__copyright__ = "Copyright 2016, Intel Corporation"
> > +__credits__ = ["Joshua Lock"]
> > +__license__ = "GPL"
> > +__version__ = "2.0"
> > +__maintainer__ = "Joshua Lock"
> > +__email__ = "joshua.g.l...@intel.com"
> > +'''
> > +
> > +# We'd probably benefit from using some caching, but first we'd
> > need the AB API
> > +# to include
> > +#
> > +# We can set repo url, branch & commit for a bunch of repositorys.
> > +# Do they all get built for nightly?
> > +
> > +try:
> > +import configparser
> > +except ImportError:
> > +import ConfigParser as configparser
> > +import json
> > +import os
> > +import requests
> > +import signal
> > +import sys
> > +import time
> > +
> > +abapi = "https://autobuilder.yoctoproject.org/main/json/builders/n
> > ightly/builds/_all"
> > +# Wiki editing params
> > +un = ''
> > +pw = ''
> > +wikiapi = "https://wiki.yoctoproject.org/wiki/api.php";
> > +title = ''
> > +
> > +last_logged = ''
> > +# TODO: probably shouldn't write files in the same location as the
> > script?
> > +cachefile = 'buildlogger.lastbuild'
> > +tmpfile = '/tmp/.buildlogger.pid'
> > +
> > +
> > +# Load configuration information from an ini
> > +def load_config(configfile):
> > +global un
> > +global pw
> > +global title
> > +success = False
> > +
> > +if os.path.exists(configfile):
> > +try:
> > +config = configparser.ConfigParser()
> > +config.read(configfile)
> > +un = config.get('wikiuser', 'username')
> > +pw = config.get('wikiuser', 'password')
> > +title = config.get('wiki', 'pagetitle')
> > +success = True
> > +except configparser.Error as ex:
> > +print('Failed to load buildlogger configuration with
> > error: %s' % str(ex))
> > +else:
> > +print('Config file %s does not exist, please create and
> > populate it.' % configfile)
> > +
> > +return success
> > +
> > +# we can't rely on the built in JSON parser in the requests module
> > because
> > +# the JSON we get from the wiki begins with a UTF-8 BOM which
> > chokes
> > +# json.loads().
> > +# Thus we decode the raw resonse content into a string and load
> > that into a
> > +# JSON object ourselves.
> > +#
> > +# http://en.wikipedia.org/wiki/Byte_Order_Mark
> > +# http://bugs.python.org/issue18958
> > +def parse_json(response):
> > +text = response.content.decode('utf-8-sig')
> > +
> > +return json.loads(text)
> > +
> > +
> > +# Get the current content of the BuildLog page -- to make the wiki
> > page as
> > +# useful as possible the most recent log entry should be at the
> > top, to
> > +# that end we need to edit the whole page so that we can insert
> > the new entry
> > +# after the log but before the other entries.
> > +# This method fetches the current page content, splits out the
> > blurb and
> > +# returns a pair:
> > +# 1) the blurb
> > +# 2) the current entries
> > +def wiki_get_content():
> > +params =
> > '?format=json&action=query&prop=revisions&rvprop=content&titles='
[yocto] [yocto-autobuilder][PATCH v2] bin/buildlogger: add new script to aid SWAT process
buildlogger will be started with the autobuilder and, when correctly
configured, monitor the AB's JSON API for newly started builds. When one is
detected information about the build will be posted to the wiki.
Requires a ConfigParser (ini) style configuration file at
AB_BASE/etc/buildlogger.conf formatted as follows:
[wikiuser]
username = botuser
password = botuserpassword
[wiki]
pagetitle = BuildLog
Signed-off-by: Joshua Lock
---
Changes since v1:
* Add example buildlogger conf file
* Only start buildlogger when BUILDLOG_TO_WIKI is True
* Move the wiki and builder api url's to the config file
.gitignore | 2 +
bin/buildlogger | 277
config/autobuilder.conf.example | 2 +
etc/buildlogger.conf.example| 8 ++
yocto-start-autobuilder | 9 ++
yocto-stop-autobuilder | 45 ---
6 files changed, 324 insertions(+), 19 deletions(-)
create mode 100755 bin/buildlogger
create mode 100644 etc/buildlogger.conf.example
diff --git a/.gitignore b/.gitignore
index 3f9505b..48c8a85 100644
--- a/.gitignore
+++ b/.gitignore
@@ -8,6 +8,7 @@
###
buildset-config
config/autobuilder.conf
+etc/buildlogger.conf
# Everything else #
###
@@ -25,6 +26,7 @@ yocto-controller/controller.cfg
yocto-controller/state.sqlite
yocto-controller/twistd.log*
yocto-controller/buildbot.tac
+yocto-controller/logger.log
yocto-worker/build-appliance/build(newcommits)
yocto-worker/buildbot.tac
yocto-worker/janitor.log
diff --git a/bin/buildlogger b/bin/buildlogger
new file mode 100755
index 000..635d55f
--- /dev/null
+++ b/bin/buildlogger
@@ -0,0 +1,277 @@
+#!/usr/bin/env python3
+'''
+Created on May 5, 2016
+
+__author__ = "Joshua Lock"
+__copyright__ = "Copyright 2016, Intel Corporation"
+__credits__ = ["Joshua Lock"]
+__license__ = "GPL"
+__version__ = "2.0"
+__maintainer__ = "Joshua Lock"
+__email__ = "joshua.g.l...@intel.com"
+'''
+
+# We'd probably benefit from using some caching, but first we'd need the AB API
+# to include
+#
+# We can set repo url, branch & commit for a bunch of repositorys.
+# Do they all get built for nightly?
+
+try:
+import configparser
+except ImportError:
+import ConfigParser as configparser
+import json
+import os
+import requests
+import signal
+import sys
+import time
+
+abapi = ''
+# Wiki editing params
+un = ''
+pw = ''
+wikiapi = ''
+title = ''
+
+last_logged = ''
+# TODO: probably shouldn't write files in the same location as the script?
+cachefile = 'buildlogger.lastbuild'
+tmpfile = '/tmp/.buildlogger.pid'
+
+
+# Load configuration information from an ini
+def load_config(configfile):
+global un
+global pw
+global title
+global wikiapi
+global abapi
+success = False
+
+if os.path.exists(configfile):
+try:
+config = configparser.ConfigParser()
+config.read(configfile)
+un = config.get('wiki', 'username')
+pw = config.get('wiki', 'password')
+title = config.get('wiki', 'pagetitle')
+wikiapi = config.get('wiki', 'apiuri')
+abapi = config.get('builder', 'apiuri')
+success = True
+except configparser.Error as ex:
+print('Failed to load buildlogger configuration with error: %s' %
str(ex))
+else:
+print('Config file %s does not exist, please create and populate it.'
% configfile)
+
+return success
+
+# we can't rely on the built in JSON parser in the requests module because
+# the JSON we get from the wiki begins with a UTF-8 BOM which chokes
+# json.loads().
+# Thus we decode the raw resonse content into a string and load that into a
+# JSON object ourselves.
+#
+# http://en.wikipedia.org/wiki/Byte_Order_Mark
+# http://bugs.python.org/issue18958
+def parse_json(response):
+text = response.content.decode('utf-8-sig')
+
+return json.loads(text)
+
+
+# Get the current content of the BuildLog page -- to make the wiki page as
+# useful as possible the most recent log entry should be at the top, to
+# that end we need to edit the whole page so that we can insert the new entry
+# after the log but before the other entries.
+# This method fetches the current page content, splits out the blurb and
+# returns a pair:
+# 1) the blurb
+# 2) the current entries
+def wiki_get_content():
+params = '?format=json&action=query&prop=revisions&rvprop=content&titles='
+req = requests.get(wikiapi+params+title)
+parsed = parse_json(req)
+pageid = sorted(parsed['query']['pages'].keys())[-1]
+content = parsed['query']['pages'][pageid]['revisions'][0]['*']
+blurb, entries = content.split('==', 1)
+# ensure we keep only a single newline after the blurb
+blurb = blurb.strip() + "\n"
+entries = '=='+entries
+
+return blurb, entries
+
+
+# Login to the wiki and return cookies for the logged in session
+def wiki_login():
+payloa
Re: [yocto] [yocto-autobuilder][PATCH v2] bin/buildlogger: add new script to aid SWAT process
Thanks, pushed to master, currently working on getting it into production.
-b
On 9 May 2016 at 13:30, Joshua Lock wrote:
> buildlogger will be started with the autobuilder and, when correctly
> configured, monitor the AB's JSON API for newly started builds. When one is
> detected information about the build will be posted to the wiki.
>
> Requires a ConfigParser (ini) style configuration file at
> AB_BASE/etc/buildlogger.conf formatted as follows:
>
> [wikiuser]
> username = botuser
> password = botuserpassword
>
> [wiki]
> pagetitle = BuildLog
>
> Signed-off-by: Joshua Lock
> ---
> Changes since v1:
> * Add example buildlogger conf file
> * Only start buildlogger when BUILDLOG_TO_WIKI is True
> * Move the wiki and builder api url's to the config file
>
> .gitignore | 2 +
> bin/buildlogger | 277
>
> config/autobuilder.conf.example | 2 +
> etc/buildlogger.conf.example| 8 ++
> yocto-start-autobuilder | 9 ++
> yocto-stop-autobuilder | 45 ---
> 6 files changed, 324 insertions(+), 19 deletions(-)
> create mode 100755 bin/buildlogger
> create mode 100644 etc/buildlogger.conf.example
>
> diff --git a/.gitignore b/.gitignore
> index 3f9505b..48c8a85 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -8,6 +8,7 @@
> ###
> buildset-config
> config/autobuilder.conf
> +etc/buildlogger.conf
>
> # Everything else #
> ###
> @@ -25,6 +26,7 @@ yocto-controller/controller.cfg
> yocto-controller/state.sqlite
> yocto-controller/twistd.log*
> yocto-controller/buildbot.tac
> +yocto-controller/logger.log
> yocto-worker/build-appliance/build(newcommits)
> yocto-worker/buildbot.tac
> yocto-worker/janitor.log
> diff --git a/bin/buildlogger b/bin/buildlogger
> new file mode 100755
> index 000..635d55f
> --- /dev/null
> +++ b/bin/buildlogger
> @@ -0,0 +1,277 @@
> +#!/usr/bin/env python3
> +'''
> +Created on May 5, 2016
> +
> +__author__ = "Joshua Lock"
> +__copyright__ = "Copyright 2016, Intel Corporation"
> +__credits__ = ["Joshua Lock"]
> +__license__ = "GPL"
> +__version__ = "2.0"
> +__maintainer__ = "Joshua Lock"
> +__email__ = "joshua.g.l...@intel.com"
> +'''
> +
> +# We'd probably benefit from using some caching, but first we'd need the AB
> API
> +# to include
> +#
> +# We can set repo url, branch & commit for a bunch of repositorys.
> +# Do they all get built for nightly?
> +
> +try:
> +import configparser
> +except ImportError:
> +import ConfigParser as configparser
> +import json
> +import os
> +import requests
> +import signal
> +import sys
> +import time
> +
> +abapi = ''
> +# Wiki editing params
> +un = ''
> +pw = ''
> +wikiapi = ''
> +title = ''
> +
> +last_logged = ''
> +# TODO: probably shouldn't write files in the same location as the script?
> +cachefile = 'buildlogger.lastbuild'
> +tmpfile = '/tmp/.buildlogger.pid'
> +
> +
> +# Load configuration information from an ini
> +def load_config(configfile):
> +global un
> +global pw
> +global title
> +global wikiapi
> +global abapi
> +success = False
> +
> +if os.path.exists(configfile):
> +try:
> +config = configparser.ConfigParser()
> +config.read(configfile)
> +un = config.get('wiki', 'username')
> +pw = config.get('wiki', 'password')
> +title = config.get('wiki', 'pagetitle')
> +wikiapi = config.get('wiki', 'apiuri')
> +abapi = config.get('builder', 'apiuri')
> +success = True
> +except configparser.Error as ex:
> +print('Failed to load buildlogger configuration with error: %s'
> % str(ex))
> +else:
> +print('Config file %s does not exist, please create and populate
> it.' % configfile)
> +
> +return success
> +
> +# we can't rely on the built in JSON parser in the requests module because
> +# the JSON we get from the wiki begins with a UTF-8 BOM which chokes
> +# json.loads().
> +# Thus we decode the raw resonse content into a string and load that into a
> +# JSON object ourselves.
> +#
> +# http://en.wikipedia.org/wiki/Byte_Order_Mark
> +# http://bugs.python.org/issue18958
> +def parse_json(response):
> +text = response.content.decode('utf-8-sig')
> +
> +return json.loads(text)
> +
> +
> +# Get the current content of the BuildLog page -- to make the wiki page as
> +# useful as possible the most recent log entry should be at the top, to
> +# that end we need to edit the whole page so that we can insert the new entry
> +# after the log but before the other entries.
> +# This method fetches the current page content, splits out the blurb and
> +# returns a pair:
> +# 1) the blurb
> +# 2) the current entries
> +def wiki_get_content():
> +params =
> '?format=json&action=query&prop=revisions&rvprop=content&titles='
> +req = requests.get(wikiapi+params+title)
> +parsed = parse_
Re: [yocto] Using multiple versions of package (PACKAGECONFIG) in the same bitbake
Thanks,
What I winded up doing is using bbappend to remove fstab from base-files. I
then created two separate packages with different fstab files, and added
each of them into their own image.
On Sun, May 8, 2016 at 1:09 PM Khem Raj wrote:
> On May 8, 2016, at 6:50 AM, Paul Knopf wrote:
>
> I have two rootfs. My main image, and the initramfs image for the kernel
> (INITRAMFS_IAMGE and INITRAMFS_IMAGE_BUNDLE).
>
> I need to provide different fstab files for each build.
>
> I have added a .bbappend for the base-files recipe to use a customized
> fstab.
>
> In the bbappend file, I'd like to add PACKAGECONFIG support to the package
> to customize it for each rootfs.
>
> However, it seems that I can only PACKAGECONFIG for a recipe in the
> local.conf. How do I add features to a package at a "image-recipe.bb"
> level?
>
> Maybe I should create a separate recipe all together that overwrite fstab?
> Then, I can include this "base-files-overrite-fstab" in the appropriate
> image recipe?
>
>
> you can write a shell function doing this in image recipe and then hook it
> up to tweak the rootfs after its created during image build.
>
> your_func() {
> …..
> }
>
> ROOTFS_POSTPROCESS_COMMAND += “your_functon;”
>
>
>
> Thanks,
> Paul Knopf
> Software Engineer
> Med X Change, Inc
> pkn...@medxchange.com
>
> --
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>
> --
Thanks,
Paul Knopf
Software Engineer
Med X Change, Inc
pkn...@medxchange.com
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
[yocto] Recipe that creates empty /var/log directory is auto-mounted to /var/volatile/log
I have a simple recipe that places files on the system.
-
do_install() {
# create this directory, because our "normal" fstab file will mount a log
partition to it.
install -d ${D}/var/log
install -d ${D}${sysconfdir}
install -m 0644 ${WORKDIR}/fstab-normal ${D}${sysconfdir}/fstab
}
-
The fstab-normal file has an entry that mounts a partition to this created
/var/log directory.
-
/dev/mmcblk0p3 /var/log auto defaults 0 0
-
However, when the package get's created, /var/log gets pointed to
/var/volatile/log. Where is this happening, and why?
-
pknopf@ubuntu:~/Git/recipes/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/base-files-fstab-normal/1.0-r0/package$
tree
.
├── etc
│ └── fstab
└── var
├── log -> volatile/log
└── volatile
└── log
-
Thanks,
Paul Knopf
Software Engineer
Med X Change, Inc
pkn...@medxchange.com
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
[yocto] Eclipse, CMake and EXTRA_OECONF_GCC_FLOAT
Hi,
I'm having some issues using the Eclipse CMake support with my own hard
float arm toolchain.
This used to work, and I believe the main change affecting my problem is
this:
https://git.yoctoproject.org/cgit.cgi/poky/commit/meta/recipes-devtools/gcc/gcc-configure-common.inc?id=075e2884ce10094258af6f7c6db1af7579b5bffa
The problem I'm having relates to the float abi. For Autotools, as far
as I can tell, the float abi is set explicitly in via the CC/CPP
variables but this is filtered out in the corresponding CMake variable,
OECMAKE_CXX_COMPILER and/or OECMAKE_CXX_FLAGS. Code in
YoctoSDKCMakeProjectNature.java (around lines 47-61):
---snip---
String ccString = YoctoSDKUtils.getEnvValue(project, "CC");
if (!ccString.equals("") && !ccString.equals(" ")) {
ccString.trim();
ccString = ccString.split(" ")[0];
}
env.addVariable("OECMAKE_C_COMPILER", ccString,
IEnvironmentVariable.ENVVAR_REPLACE, delimiter, ccdesc);
String cxxString = YoctoSDKUtils.getEnvValue(project, "CXX");
if (!cxxString.equals("") && !cxxString.equals(" ")) {
cxxString.trim();
cxxString = cxxString.split(" ")[0];
}
env.addVariable("OECMAKE_CXX_COMPILER", cxxString,
IEnvironmentVariable.ENVVAR_REPLACE, delimiter, ccdesc);
---snip---
And the removed part is not added back to any other variable. So this
used to work because before the value of
EXTRA_OECONF_GCC_FLOAT/EXTRA_OECONF_FPU would be such that the hard
float abi would be the default for hard float MACHINEs. After the commit
linked above it isn't, so the abi needs to be set explicitly when
calling the compiler. But, it seems those flags are not carried over to
CMake in Eclipse.
Is this a bug with the CMake support, or should I actually set
EXTRA_OECONF_GCC_FLOAT="--with-float=hard"?
Changing the variables seems to be exactly what the commit wanted to
avoid so both hard and soft float machines can use the same cross toolchain.
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
[yocto] How to build a simple kernel modul
Hej,
I want to build a recipe for a simple GPIO Module. I found no tutorial at
the documentation. What is the comon way to build a simple kernel module?
The Makefile looks like:
#
obj-m += ledmodule.o
SRC := $(shell pwd)
all: modules
modules:
$(MAKE) -C $(KERNEL_SRC) M=$(SRC) modules
modules_install:
$(MAKE) -C $(KERNEL_SRC) M=$(SRC) modules_install
clean:
rm -rf *.o *.ko *.mod.c
#
The recipe contains:
#
SUMMARY = "ledmodule"
SECTION = ""
LICENSE = "CLOSED"
inherit module
SRC_URI = "file://ledmodule.c \
file://Makefile \
"
S = "${WORKDIR}/ledmodule-0.1"
#
Regards
Stefan Jaritz
Entwickler
ESA Elektroschaltanlagen Grimma GmbH
Broner Ring 30
04668 Grimma
Telefon: +49 3437 9211 176
Telefax: +49 3437 9211 26
E-Mail: s.jar...@esa-grimma.de
Internet: www.esa-grimma.de
Geschäftsführer:
Dipl.-Ing. Jörg Gaitzsch
Jörg Reinker
Sitz der Gesellschaft: Grimma
Ust.-ID: DE 141784437
Amtsgericht: Leipzig, HRB 5159
Steuernummer: 238/108/00755
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte
Informationen.
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich
erhalten
haben, informieren Sie bitte sofort den Absender und löschen Sie diese
Nachricht. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser
Mail
ist nicht gestattet.
This e-mail may contain confidential and/or privileged information. If you
are
not the intended recipient (or have received this e-mail in error) please
notify the sender immediately and destroy this e-mail. Any unauthorized
copying, disclosure or distribution of the material in this e-mail is
strictly
forbidden.--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Recipe that creates empty /var/log directory is auto-mounted to /var/volatile/log
It happens in meta/recipes-core/initscripts. I don't know the history behind
it, but the OE convention is to use /var/volatile, I believe mainly so that
/var/volatile could be a RAMFS.
-Bill
On Mon, 2016-05-09 at 10:02 -0400, Paul Knopf wrote:
I have a simple recipe that places files on the system.
-
do_install() {
# create this directory, because our "normal" fstab file will mount a log
partition to it.
install -d ${D}/var/log
install -d ${D}${sysconfdir}
install -m 0644 ${WORKDIR}/fstab-normal ${D}${sysconfdir}/fstab
}
-
The fstab-normal file has an entry that mounts a partition to this created
/var/log directory.
-
/dev/mmcblk0p3 /var/log auto defaults 0 0
-
However, when the package get's created, /var/log gets pointed to
/var/volatile/log. Where is this happening, and why?
-
pknopf@ubuntu:~/Git/recipes/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/base-files-fstab-normal/1.0-r0/package$
tree
.
├── etc
│ └── fstab
└── var
├── log -> volatile/log
└── volatile
└── log
-
Thanks,
Paul Knopf
Software Engineer
Med X Change, Inc
pkn...@medxchange.com
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
[yocto] [ANNOUNCEMENT] Chameleonsocks 1.0 is now available
Chameleonsocks provides containerized system-wide redsocks-based TCP redirector to generic SOCKS or HTTP proxies. It is particularly useful for people who use Yocto on firewalled networks where the real Internet is accessed through proxies. The installer script can be downloaded from the upstream repository: https://github.com/crops/chameleonsocks This release includes the following functionality: --install : Install chameleonsocks --upgrade : Upgrade existing installation --uninstall: Uninstall chameleonsocks --install-ui : Install container management UI --uninstall-ui : Uninstall container management UI --start: Stop chameleonsocks --stop : Start chameleonsocks --version : Display chameleonsocks version Todor Minchev Intel Open Source Technology Center -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Recipe that creates empty /var/log directory is auto-mounted to /var/volatile/log
Some people choose a read-only file system. Thus /var/log is not writable.
However, some software expects to use the hard coded path of /var/log.
Thus, there's a symlink to the ramdisk to solve both problems.
From: yocto-boun...@yoctoproject.org on behalf
of Paul Knopf
Sent: Monday, May 9, 2016 7:02 AM
To: yocto@yoctoproject.org
Subject: [yocto] Recipe that creates empty /var/log directory is auto-mounted
to /var/volatile/log
I have a simple recipe that places files on the system.
-
do_install() {
# create this directory, because our "normal" fstab file will mount a log
partition to it.
install -d ${D}/var/log
install -d ${D}${sysconfdir}
install -m 0644 ${WORKDIR}/fstab-normal ${D}${sysconfdir}/fstab
}
-
The fstab-normal file has an entry that mounts a partition to this created
/var/log directory.
-
/dev/mmcblk0p3 /var/log auto defaults 0 0
-
However, when the package get's created, /var/log gets pointed to
/var/volatile/log. Where is this happening, and why?
-
pknopf@ubuntu:~/Git/recipes/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/base-files-fstab-normal/1.0-r0/package$
tree
.
??? etc
? ??? fstab
??? var
??? log -> volatile/log
??? volatile
??? log
-
Thanks,
Paul Knopf
Software Engineer
Med X Change, Inc
pkn...@medxchange.com
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [PATCH v2 1/2] kernel-dev: Fix the locations of .config and source directory
ping
On 05/03/2016 11:28 AM, Tom Zanussi wrote:
The locations of the kernel .config file and source directory moved a
couple releases ago; update the documentation accordingly.
Also, add a note explaining how to expand variables, which serves a
couple of purposes:
* For curious readers, shows them how to demystify where these
variables come from and how they're used.
* For suspicious readers, shows them how they can verify that the
variables in the documentation are actually correct.
Fixes [YOCTO #9124].
Signed-off-by: Tom Zanussi
---
documentation/kernel-dev/kernel-dev-common.xml | 20 +++-
1 file changed, 15 insertions(+), 5 deletions(-)
diff --git a/documentation/kernel-dev/kernel-dev-common.xml
b/documentation/kernel-dev/kernel-dev-common.xml
index 261471c..5bd12be 100644
--- a/documentation/kernel-dev/kernel-dev-common.xml
+++ b/documentation/kernel-dev/kernel-dev-common.xml
@@ -384,9 +384,7 @@
The resulting .config file is
-located in
-${WORKDIR}
under the
-linux-${PACKAGE_ARCH}-${LINUX_KERNEL_TYPE}-build directory.
+located in the build directory, ${B}, which expands to ${WORKDIR}/linux-${PACKAGE_ARCH}-${LINUX_KERNEL_TYPE}-build.
You can use the entire .config file as
the
defconfig file as described in the
"Changing the
Configuration" section.
@@ -396,6 +394,19 @@
section in the Yocto Project Development Manual.
+
+ If you're ever confused about what a variable expands
+ to, you can determine that unequivocally by looking at
+ the output of the bitbake -e command:
+
+
+ $ bitbake -e virtual/kernel
+
+
+ Search the output for the variable you're interested in
+ and you can see exactly how it's expanded and used.
+
+
A better method is to create a configuration fragment using
the
differences between two configuration files: one previously
@@ -512,8 +523,7 @@
Taking this step ensures you have the sources prepared
and the configuration completed.
-You can find the sources in the
-${WORKDIR}/linux
directory.
+You can find the sources in the symlinked source/ directory just off the build directory, in other words ${B}/source, which expands to
${WORKDIR}/linux-${PACKAGE_ARCH}-${LINUX_KERNEL_TYPE}-build/source. The directory pointed to by the
source/ symlink is also known as ${STAGING_KERNEL_DIR}
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
[linux-yocto] [PATCH] meta-yocto-bsp: beaglebone: Enable drm for omap
From: Edwin Plauchu The modesettings driver for beaglebone is requiring this module enable. --- bsp/beaglebone/beaglebone.cfg | 1 + 1 file changed, 1 insertion(+) diff --git a/bsp/beaglebone/beaglebone.cfg b/bsp/beaglebone/beaglebone.cfg index 781ecfc..66a6361 100644 --- a/bsp/beaglebone/beaglebone.cfg +++ b/bsp/beaglebone/beaglebone.cfg @@ -219,6 +219,7 @@ CONFIG_FB=y CONFIG_DRM=y CONFIG_DRM_I2C_NXP_TDA998X=y CONFIG_DRM_TILCDC=y +CONFIG_DRM_OMAP=y # CONFIG_VGASTATE is not set # CONFIG_VIDEO_OUTPUT_CONTROL is not set # CONFIG_FIRMWARE_EDID is not set -- 1.9.1 -- ___ linux-yocto mailing list linux-yo...@yoctoproject.org https://lists.yoctoproject.org/listinfo/linux-yocto
Re: [yocto] How to build a simple kernel modul
2016-05-09 17:06 skrev s.jar...@esa-grimma.de:
> Hej,
>
> I want
to build a recipe for a simple GPIO Module. I found no tutorial at the
documentation. What is the comon way to build a simple kernel module?
>
> The Makefile looks like:
> #
> obj-m += ledmodule.o
>
>
SRC := $(shell pwd)
>
> all: modules
>
> modules:
> $(MAKE) -C
$(KERNEL_SRC) M=$(SRC) modules
>
> modules_install:
> $(MAKE) -C
$(KERNEL_SRC) M=$(SRC) modules_install
>
> clean:
> rm -rf *.o *.ko
*.mod.c
> #
> The recipe contains:
> #
> SUMMARY =
"ledmodule"
> SECTION = ""
> LICENSE = "CLOSED"
>
> inherit module
>
> SRC_URI = "file://ledmodule.c [1]
> file://Makefile [2]
> "
> S
= "${WORKDIR}/ledmodule-0.1"
> #
>
> Regards
>
> Stefan
Jaritz
> Entwickler
>
>
> ESA
Elektroschaltanlagen Grimma GmbH
> Broner Ring 30
> 04668 Grimma
>
Telefon: +49 3437 9211 176
> Telefax: +49 3437 9211 26
> E-Mail:
s.jar...@esa-grimma.de
> Internet: www.esa-grimma.de
>
>
Geschäftsführer:
> Dipl.-Ing. Jörg Gaitzsch
> Jörg Reinker
>
> Sitz der
Gesellschaft: Grimma
> Ust.-ID: DE 141784437
> Amtsgericht: Leipzig, HRB
5159
> Steuernummer: 238/108/00755
>
> Diese E-Mail enthält
vertrauliche und/oder rechtlich geschützte Informationen.
> Wenn Sie
nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten
> haben, informieren Sie bitte sofort den Absender und löschen Sie
diese
> Nachricht. Das unerlaubte Kopieren sowie die unbefugte
Weitergabe dieser Mail
> ist nicht gestattet.
>
> This e-mail may
contain confidential and/or privileged information. If you are
> not
the intended recipient (or have received this e-mail in error) please
>
notify the sender immediately and destroy this e-mail. Any unauthorized
> copying, disclosure or distribution of the material in this e-mail is
strictly
> forbidden.
Hi,
There are actually some guides for this,
see lab4
in:
https://www.yoctoproject.org/sites/default/files/kernel-lab-2.1_0.pdf
But to simply build a module using bitbake, you can just do:
$
bitbake ledmodule
then you will find your recipe and the module in the
./tmp/work/ structure (assuming you are not inheriting 'rm_work' in
local.conf...).
if you just want to compile it, you can run:
$
bitbake ledmodule -c compile
BR Petter
Petter Mabäcker
Technux
www.technux.se
Links:
--
[1]
file://sek4_ledmodule.c/
[2] file://Makefile/
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
