On Tue, May 24, 2016 at 10:45 AM, Christopher Larson <kerg...@gmail.com>
wrote:
>
> On Tue, May 24, 2016 at 9:07 AM, Christopher Larson <kerg...@gmail.com>
> wrote:
>
>>
>> On Tue, May 24, 2016 at 7:26 AM, Christopher Larson <kerg...@gmail.com>
>> wrote:
>>
>>>
>>> On Tue, May 24, 2016 at 2:13 AM, Richard Purdie <
>>> richard.pur...@linuxfoundation.org> wrote:
>>>
>>>> On Mon, 2016-05-23 at 13:34 -0700, Christopher Larson wrote:
>>>> > From: Christopher Larson <chris_lar...@mentor.com>
>>>> >
>>>> > These files are treated as the contents of a bitbake variable, so
>>>> > usual
>>>> > bitbake variable references are supported. I considered using another
>>>> > templating mechanism, for example the one used by yocto-layer, but
>>>> > then we'd
>>>> > end up largely mapping metadata variables to template fields anyway,
>>>> > which is
>>>> > a pointless indirection. Let bitbake expand the variables directly
>>>> > instead.
>>>> >
>>>> > This feature lets us, for example, reference ${APPEND} in --append,
>>>> > and avoid
>>>> > hardcoding the serial console tty in the wks file, and let the user's
>>>> > changes
>>>> > to APPEND affect wic the way they do the other image construction
>>>> > mechanisms.
>>>> >
>>>> > The template is read in and set in a variable at parse time, so
>>>> > changes to the
>>>> > variables referenced by the template will result in rebuilding the
>>>> > image.
>>>> >
>>>> > Signed-off-by: Christopher Larson <chris_lar...@mentor.com>
>>>> > ---
>>>> > meta/classes/image_types.bbclass | 35
>>>> > +++++++++++++++++++++++++++++++++++
>>>> > 1 file changed, 35 insertions(+)
>>>> >
>>>> > diff --git a/meta/classes/image_types.bbclass
>>>> > b/meta/classes/image_types.bbclass
>>>> > index dc681ae..caf8757 100644
>>>> > --- a/meta/classes/image_types.bbclass
>>>> > +++ b/meta/classes/image_types.bbclass
>>>> > @@ -206,6 +206,16 @@ IMAGE_CMD_wic () {
>>>> > }
>>>> > IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
>>>> >
>>>> > +python process_wks_template () {
>>>> > + """Write out expanded template contents to WKS_FULL_PATH."""
>>>> > + template_body = d.getVar('_WKS_TEMPLATE', True)
>>>> > + if template_body:
>>>> > + wks_file = d.getVar('WKS_FULL_PATH', True)
>>>> > + with open(wks_file, 'w') as f:
>>>> > + f.write(template_body)
>>>> > +}
>>>> > +do_image_wic[prefuncs] += 'process_wks_template'
>>>> > +
>>>> > # Rebuild when the wks file or vars in WICVARS change
>>>> > USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + '
>>>> > '.join('wic.%s' % c for c in '${COMPRESSIONTYPES}'.split()), '1', '',
>>>> > d)}"
>>>> > WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
>>>> > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
>>>> > @@ -302,3 +312,28 @@ IMAGE_TYPES_MASKED ?= ""
>>>> > # The WICVARS variable is used to define list of bitbake variables
>>>> > used in wic code
>>>> > # variables from this list is written to <image>.env file
>>>> > WICVARS ?= "BBLAYERS DEPLOY_DIR_IMAGE HDDDIR IMAGE_BASENAME
>>>> > IMAGE_BOOT_FILES IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES
>>>> > INITRD ISODIR MACHINE_ARCH ROOTFS_SIZE STAGING_DATADIR
>>>> > STAGING_DIR_NATIVE STAGING_LIBDIR TARGET_SYS"
>>>> > +
>>>> > +python () {
>>>> > + """Read in and set up wks file template for wic."""
>>>> > + if d.getVar('USING_WIC', True):
>>>> > + wks_file_u = d.getVar('WKS_FULL_PATH', False)
>>>> > + wks_file = d.expand(wks_file_u)
>>>> > + base, ext = os.path.splitext(wks_file)
>>>> > + if ext == '.in' and os.path.exists(wks_file):
>>>> > + wks_out_file = os.path.join(d.getVar('WORKDIR', True),
>>>> > os.path.basename(base))
>>>> > + d.setVar('WKS_FULL_PATH', wks_out_file)
>>>> > + d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
>>>> > + d.setVar('WKS_FILE_CHECKSUM',
>>>> > '${WKS_TEMPLATE_PATH}:True')
>>>> > +
>>>> > + try:
>>>> > + with open(wks_file, 'r') as f:
>>>> > + body = f.read()
>>>> > + except (IOError, OSError) as exc:
>>>> > + pass
>>>>
>>>> I'm a little nervous about determinism here. Shouldn't it either find
>>>> the referenced file or error?
>>>
>>>
>>> That's a fair point, I think I was concerned about parse time errors,
>>> since they tend to blow up into a lot of unpleasantness for the user, since
>>> parse time errors in classes can cause errors for multiple recipes. If
>>> no file is read in at parse time, there'll be no contents to write out,
>>> and we could then error when trying to build the image with wic, as no
>>> wks file will exist.
>>>
>>> Of course, the file being missing will change checksums for the image
>>> tasks, but I think that's reasonable.
>>>
>>> Thoughts?
>>>
>>
>> I'll do a bit more testing on the behaviors and get back to you on that
>> later. Thanks.
>
>
> If the WKS_FILE doesn't exist, whether we're using templates or not, it'll
> fail in do_image_wic:
>
> ERROR: core-image-sato-1.0-r0 do_image_wic: No kickstart files from
> WKS_FILES were found: /scratch/dogwood/minnowmax/build/mkefidiskf.wks.in
> core-image-sato.wks. Please set WKS_FILE or WKS_FILES appropriately.
> ERROR: core-image-sato-1.0-r0 do_image_wic: Function failed: do_image_wic
> (log file is located at
> /scratch/dogwood/minnowmax/build/tmp/work/intel_corei7_64-mel-linux/core-image-sato/1.0-r0/temp/log.do_image_wic.12181)
>
> If the template doesn't exist, then no template handling will be done at
> all, so do_image_wic won't depend on any of the variables referenced in the
> template, and the task will fail. If it does, then we will depend on its
> contents and referenced variables, and the task will not fail.
>
> I don't really see a problem with determinism there. Either the file
> exists at parse time or it doesn't, and the behavior seems reasonable in
> either case.
>
Update: I've occasionally seen a taskhash mismatch, so clearly I've missed
something. Hold off on this until further notice. Thanks again for the
feedback.
--
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
