Thanks - I have a preliminary fix here (needs thorough testing):
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=akanavin/package-version-updates&id=d6b6a6f67cc967c0ead6dbab3c95435b7ca85246
Alex
On Thu, 12 Nov 2020 at 19:46, Glenn Stroz <gst...@uber.com> wrote:
> Hi Alex,
>
> We have a custom fetcher implemented as a bbclass that calls out to a
> python script. That is why we see these issues with native. Here's a
> sample bbclass and recipe that reproduces the issue:
>
> impfail.bbclass
> ```
> inherit python3native
>
> do_unpack[noexec] = "1"
> do_patch[noexec] = "1"
> do_install[noexec] = "1"
>
> do_fetch() {
> python3 -c "import pydoc"
> }
> ```
>
> classrunner_0.0.1.bb
> ```
> SUMMARY = "Python runner"
> LICENSE = "CLOSED"
> inherit impfail
> ```
>
> -Glenn
>
>
> On Thu, Nov 12, 2020 at 4:52 AM Alexander Kanavin <alex.kana...@gmail.com>
> wrote:
>
>> A quick fix may be to add a target python3 dependency to your recipe, yes
>> ugly, but I can't figure out how to provide target python configuration to
>> native python used in the context of building target recipes otherwise.
>>
>> Alex
>>
>>
>> On Thu, 12 Nov 2020 at 10:49, Alexander Kanavin <alex.kana...@gmail.com>
>> wrote:
>>
>>> On Thu, 12 Nov 2020 at 02:23, Glenn Stroz via lists.openembedded.org
>>> <gstroz=uber....@lists.openembedded.org> wrote:
>>>
>>>> We've been upgrading our work to Dunfell and have encountered an issue
>>>> with python3-native.
>>>>
>>>> One of our python3-native scripts uses pydoc and that in turn
>>>> eventually pulls in python's sysconfig. However, this is failing to import
>>>> with the following stack trace:
>>>>
>>>> ```
>>>> import pydoc
>>>> File
>>>> "/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python3.8/pydoc.py",
>>>> line 366, in <module>
>>>> class Doc:
>>>> File
>>>> "/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python3.8/pydoc.py",
>>>> line 396, in Doc
>>>> def getdocloc(self, object, basedir=sysconfig.get_path('stdlib')):
>>>> File
>>>> "/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python3.8/sysconfig.py",
>>>> line 519, in get_path
>>>> return get_paths(scheme, vars, expand)[name]
>>>> File
>>>> "/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python3.8/sysconfig.py",
>>>> line 509, in get_paths
>>>> return _expand_vars(scheme, vars)
>>>> File
>>>> "/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python3.8/sysconfig.py",
>>>> line 172, in _expand_vars
>>>> _extend_dict(vars, get_config_vars())
>>>> File
>>>> "/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python3.8/sysconfig.py",
>>>> line 558, in get_config_vars
>>>> _init_posix(_CONFIG_VARS)
>>>> File
>>>> "/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python3.8/sysconfig.py",
>>>> line 428, in _init_posix
>>>> _temp = __import__(name, globals(), locals(), ['build_time_vars'],
>>>> 0)
>>>> ModuleNotFoundError: No module named '_sysconfigdata'
>>>> ```
>>>>
>>>> I did a little digging and the following commit seems to reference
>>>> related work:
>>>>
>>>>
>>>> https://github.com/openembedded/openembedded-core/commit/02714c105426b0d687620913c1a7401b386428b6
>>>>
>>>> I see that there is a patch added which adds the following to
>>>> `sysconfig.py`:
>>>>
>>>> ```
>>>> if 'STAGING_LIBDIR' in os.environ:
>>>>
>>>> sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata')
>>>> ```
>>>>
>>>> However, STAGING_LIBDIR does not contain this path:
>>>>
>>>> ```
>>>> ls
>>>> /home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot/usr/lib/python-sysconfigdata
>>>>
>>>> ls: cannot access
>>>> '/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot/usr/lib/python-sysconfigdata':
>>>> No such file or directory
>>>> ```
>>>>
>>>> It appears as if this should really be pointing
>>>> to STAGING_LIBDIR_NATIVE which does exist:
>>>>
>>>> ```
>>>> ls
>>>> /home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python-sysconfigdata
>>>> _sysconfigdata.py
>>>> ```
>>>>
>>>> The referenced commit made efforts to combine the native and cross
>>>> python3 bbclass files into one. However, I think this may be something that
>>>> needs to be treated differently.
>>>>
>>>> Anyone have any thoughts on how to proceed?
>>>>
>>>
>>> Can you share the exact context where this is happening? Is your script
>>> coming from a recipe and can you show that recipe?
>>>
>>> Alex
>>>
>>>
>>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#144531):
https://lists.openembedded.org/g/openembedded-core/message/144531
Mute This Topic: https://lists.openembedded.org/mt/78197661/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
