On Tue, 2026-08-04 at 08:02 +0200, Daniel Turull via lists.openembedded.org wrote: > From: Daniel Turull <[email protected]> > > bb.process.run() always decodes command output as UTF-8, which raises > UnicodeDecodeError for changelogs containing non-UTF-8 bytes (e.g. > Latin-1 author names). Add _git_show_file(), which redirects > `git show` to a temp file and reads it back with errors='replace' to > tolerate that, and use it for the existing per-version release notes > lookup in _extract_changelog(), which had the same crash exposure. > > AI-Generated: Kiro with Claude Sonnet 5 > Signed-off-by: Daniel Turull <[email protected]> > --- > scripts/lib/devtool/upgrade.py | 16 ++++++++++++++-- > 1 file changed, 14 insertions(+), 2 deletions(-) > > diff --git a/scripts/lib/devtool/upgrade.py b/scripts/lib/devtool/upgrade.py > index 495a5b8217..6883c99cce 100644 > --- a/scripts/lib/devtool/upgrade.py > +++ b/scripts/lib/devtool/upgrade.py > @@ -589,6 +589,18 @@ def _resolve_rst_includes(content, srctree): > return ''.join(result) > > > +def _git_show_file(srctree, ref, fname): > + """Return the content of fname at ref. Changelogs occasionally contain > + non-UTF-8 bytes (e.g. Latin-1 author names), which bb.process.run() > + can't handle since it always decodes command output as UTF-8; > + redirecting to a temp file and reading it back leniently avoids that > + restriction.""" > + with tempfile.NamedTemporaryFile(prefix='devtool-changelog') as tmpf: > + _run('git show %s > %s' % (shlex.quote('%s:%s' % (ref, fname)), > shlex.quote(tmpf.name)), > srctree)
The alternative to use a tmp file was to modify _run to allow other encodings than utf-8, but it will be a bigger refactor. Or not use _run and call directly the raw subprocess calls. I can change it to either option or keep the current patch. > + with open(tmpf.name, 'r', errors='replace') as f: > + return f.read() > + > + > Daniel
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#242700): https://lists.openembedded.org/g/openembedded-core/message/242700 Mute This Topic: https://lists.openembedded.org/mt/120590102/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
