Hi Patrick,
On 28.11.2016 16:33, Patrick Ohly wrote:
> /proc/[diskstats|meminfo|stat] get sampled and written to the same
> proc_<filename>.log files as during normal bootchat logging. This will
> allow rendering the CPU, disk and memory usage charts.
>
> Right now sampling happens once a second, triggered by the heartbeat
> event.That produces quite a bit of data for long builds, which will be
> addressed in a separate commit by storing the data in a more compact
> form.
>
> Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
> ---
> meta/classes/buildstats.bbclass | 16 +++++++++++++
> meta/lib/buildstats.py | 50
> +++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 66 insertions(+)
> create mode 100644 meta/lib/buildstats.py
>
> diff --git a/meta/classes/buildstats.bbclass b/meta/classes/buildstats.bbclass
> index 57ecc8f..2abc1a7 100644
> --- a/meta/classes/buildstats.bbclass
> +++ b/meta/classes/buildstats.bbclass
> @@ -188,3 +188,19 @@ python run_buildstats () {
> addhandler run_buildstats
> run_buildstats[eventmask] = "bb.event.BuildStarted bb.event.BuildCompleted
> bb.build.TaskStarted bb.build.TaskSucceeded bb.build.TaskFailed"
>
> +python runqueue_stats () {
> + import buildstats
> + from bb import event, runqueue
> + # We should not record any samples before the first task has started,
> + # because that's the first activity shown in the process chart.
> + # Besides, at that point we are sure that the build variables
> + # are available that we need to find the output directory.
> + init = isinstance(e, bb.runqueue.runQueueTaskStarted)
> + system_stats = buildstats.get_system_stats(d, init=init)
> + if system_stats:
> + # Ensure that we sample at important events.
> + system_stats.sample(force=isinstance(e, bb.event.BuildCompleted))
> +}
> +
> +addhandler runqueue_stats
> +runqueue_stats[eventmask] = "bb.runqueue.runQueueTaskStarted
> bb.event.HeartbeatEvent bb.event.BuildCompleted"
> diff --git a/meta/lib/buildstats.py b/meta/lib/buildstats.py
> new file mode 100644
> index 0000000..1664c52
> --- /dev/null
> +++ b/meta/lib/buildstats.py
> @@ -0,0 +1,50 @@
> +# Implements system state sampling. Called by buildstats.bbclass.
> +# Because it is a real Python module, it can hold persistent state,
> +# like open log files and the time of the last sampling.
> +
> +import time
> +
> +class SystemStats:
> + def __init__(self, d):
> + bn = d.getVar('BUILDNAME', True)
> + bsdir = os.path.join(d.getVar('BUILDSTATS_BASE', True), bn)
> + bb.utils.mkdirhier(bsdir)
> +
> + self.proc_files = []
> + for filename in ('diskstats', 'meminfo', 'stat'):
> + # In practice, this class gets instantiated only once in
> + # the bitbake cooker process. Therefore 'append' mode is
> + # not strictly necessary, but using it makes the class
> + # more robust should two processes ever write
> + # concurrently.
> + self.proc_files.append((filename,
> + open(os.path.join(bsdir, 'proc_%s.log' %
> filename), 'ab')))
> + # Last time that we sampled data.
> + self.last = 0
> + # Minimum number of seconds between recording a sample. This
> + # becames relevant when we get called very often while many
> + # short tasks get started. Sampling during quiet periods
> + # depends on the heartbeat event, which fires less often.
> + self.min_seconds = 1
> +
> + def sample(self, force):
> + now = time.time()
> + if (now - self.last > self.min_seconds) or force:
> + for filename, output in self.proc_files:
> + with open(os.path.join('/proc', filename), 'rb') as input:
^
this fails with a backtrace if /proc/diskstats is not available, which
may be the case in virtualized environments, e.g. Linux-VServer.
Regards,
Andreas
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
