Mike, Thanks so different encase.
Thanks, Andrew Fish > On Jun 12, 2020, at 12:25 PM, Kinney, Michael D <michael.d.kin...@intel.com> > wrote: > > Hi Andrew, > > Not related to patch vs patch set. All commits in a pr are being evaluated. > > After further evaluation, it appears it may be related to a patch set that > deletes the last file from a directory. When that happens, the parent > directory is also deleted, and the missing dir is what is generating the > python exception. Need to add logic to cover that corner case and avoid > exception. > > The edk2-pytools owners are working on it now. > > Mike > > From: Andrew Fish <af...@apple.com <mailto:af...@apple.com>> > Sent: Friday, June 12, 2020 11:43 AM > To: devel@edk2.groups.io <mailto:devel@edk2.groups.io>; Kinney, Michael D > <michael.d.kin...@intel.com <mailto:michael.d.kin...@intel.com>> > Cc: Ard Biesheuvel <ard.biesheu...@arm.com <mailto:ard.biesheu...@arm.com>>; > Sean Brogan <spbro...@outlook.com <mailto:spbro...@outlook.com>>; > ler...@redhat.com <mailto:ler...@redhat.com>; Bret Barkelew > <bret.barke...@microsoft.com <mailto:bret.barke...@microsoft.com>> > Subject: Re: [edk2-devel] ci problems again > > Mike, > > Is this an issue of assuming a patch vs. a patch set? Are there other places > we could run into this? > > Thanks, > > Andrew Fish > > > On Jun 12, 2020, at 11:19 AM, Michael D Kinney <michael.d.kin...@intel.com > <mailto:michael.d.kin...@intel.com>> wrote: > > Hi Ard, > > I think this is related to a patch series that deletes > some files. > > The following code in the edk2-pytool-extensions is collecting > the files that change in a pr to determine the components that > need to be built. > > https://github.com/tianocore/edk2-pytool-extensions/blob/e036937666c797d6d1aa4a8285088d3eae128d6c/edk2toolext/invocables/edk2_pr_eval.py#L271 > > <https://github.com/tianocore/edk2-pytool-extensions/blob/e036937666c797d6d1aa4a8285088d3eae128d6c/edk2toolext/invocables/edk2_pr_eval.py#L271> > > It does not filter out files that have been deleted. > > I tried the following change that filters out files that > have been deleted and reverses the diff to be from the > base branch to HEAD so the files deleted filter works as > expected. The stuart command completes without errors. > > diff --git a/edk2toolext/invocables/edk2_pr_eval.py > b/edk2toolext/invocables/edk2_pr_eval.py > index 00ac282..a3f55b7 100644 > --- a/edk2toolext/invocables/edk2_pr_eval.py > +++ b/edk2toolext/invocables/edk2_pr_eval.py > @@ -268,7 +268,7 @@ def _get_files_that_changed_in_this_pr(self, base_branch) > -> tuple: > > # get file differences between pr and base > output = StringIO() > - cmd_params = f"diff --name-only HEAD..{base_branch}" > + cmd_params = f"diff --name-only --diff-filter=d {base_branch}..HEAD" > rc = RunCmd("git", cmd_params, outstream=output) > > if(rc == 0): > > I do not know if this change has other unintended side effects. I > need edk2-pytools-extensions to provide input. > > Mike > > > -----Original Message----- > From: Kinney, Michael D <michael.d.kin...@intel.com > <mailto:michael.d.kin...@intel.com>> > Sent: Friday, June 12, 2020 9:49 AM > To: Ard Biesheuvel <ard.biesheu...@arm.com <mailto:ard.biesheu...@arm.com>>; > devel@edk2.groups.io <mailto:devel@edk2.groups.io>; Sean Brogan > <spbro...@outlook.com <mailto:spbro...@outlook.com>>; ler...@redhat.com > <mailto:ler...@redhat.com>; Kinney, > Michael D <michael.d.kin...@intel.com <mailto:michael.d.kin...@intel.com>> > Subject: RE: [edk2-devel] ci problems again > > Hi Ard, > > The general instructions to setup env are in the repo > here: > > > https://github.com/tianocore/edk2/tree/master/.pytool > <https://github.com/tianocore/edk2/tree/master/.pytool> > > I did this and pulled PR 689 and ran the command from > the > Azure pipelines log and reproduced the same failure on > my > laptop. Here is the log from my laptop. I will debug a > bit more to see what the issue is. This command passes > with edk2/master. It fails with my pr_689 branch. > > > (env) c:\work\GitHub\tianocore\edk2>git checkout pr_689 > Switched to branch 'pr_689' > > (env) c:\work\GitHub\tianocore\edk2>stuart_pr_eval -c > OvmfPkg/PlatformCI/PlatformBuild.py -t DEBUG -a IA32 -- > pr-target origin/master > SECTION - Init SDE > SECTION - Loading Plugins > SECTION - Start Invocable Tool > ERROR - Failed to get package for file > ArmVirtPkg/Include/Platform/Hidden.h. Exception > [WinError 3] The system cannot find the path specified: > 'c:\\work\\GitHub\\tianocore\\edk2\\ArmVirtPkg\\Include\ > \Platform' > ERROR - Failed to get package for file > ArmVirtPkg/PrePi/Scripts/PrePi-PIE.lds. Exception > [WinError 3] The system cannot find the path specified: > 'c:\\work\\GitHub\\tianocore\\edk2\\ArmVirtPkg\\PrePi\\S > cripts' > ERROR - Failed to get package for file > ArmVirtPkg/Include/Platform/Hidden.h. Exception > [WinError 3] The system cannot find the path specified: > 'c:\\work\\GitHub\\tianocore\\edk2\\ArmVirtPkg\\Include\ > \Platform' > ERROR - Failed to get package for file > ArmVirtPkg/PrePi/Scripts/PrePi-PIE.lds. Exception > [WinError 3] The system cannot find the path specified: > 'c:\\work\\GitHub\\tianocore\\edk2\\ArmVirtPkg\\PrePi\\S > cripts' > Traceback (most recent call last): > File "C:\Program Files (x86)\Microsoft Visual > Studio\Shared\Python37_64\lib\runpy.py", line 193, in > _run_module_as_main > "__main__", mod_spec) > File "C:\Program Files (x86)\Microsoft Visual > Studio\Shared\Python37_64\lib\runpy.py", line 85, in > _run_code > exec(code, run_globals) > File > "c:\work\GitHub\tianocore\edk2\env\Scripts\stuart_pr_eva > l.exe\__main__.py", line 7, in <module> > File "c:\work\github\tianocore\edk2\env\lib\site- > packages\edk2toolext\invocables\edk2_pr_eval.py", line > 389, in main > Edk2PrEval().Invoke() > File "c:\work\github\tianocore\edk2\env\lib\site- > packages\edk2toolext\base_abstract_invocable.py", line > 141, in Invoke > retcode = self.Go() > File "c:\work\github\tianocore\edk2\env\lib\site- > packages\edk2toolext\invocables\edk2_pr_eval.py", line > 82, in Go > actualPackagesDict = > self.get_packages_to_build(self.requested_package_list) > File "c:\work\github\tianocore\edk2\env\lib\site- > packages\edk2toolext\invocables\edk2_pr_eval.py", line > 202, in get_packages_to_build > changed_modules = > self._get_unique_module_infs_changed(files) > File "c:\work\github\tianocore\edk2\env\lib\site- > packages\edk2toolext\invocables\edk2_pr_eval.py", line > 235, in _get_unique_module_infs_changed > infs = > self.edk2_path_obj.GetContainingModules(os.path.abspath( > f)) > File "c:\work\github\tianocore\edk2\env\lib\site- > packages\edk2toollib\uefi\edk2\path_utilities.py", line > 211, in GetContainingModules > for f in os.listdir(dirpath): > FileNotFoundError: [WinError 3] The system cannot find > the path specified: > 'c:\\work\\GitHub\\tianocore\\edk2\\ArmVirtPkg\\Include\ > \Platform' > > Best regards, > > Mike > > > -----Original Message----- > From: Ard Biesheuvel <ard.biesheu...@arm.com <mailto:ard.biesheu...@arm.com>> > Sent: Friday, June 12, 2020 9:16 AM > To: Kinney, Michael D <michael.d.kin...@intel.com > <mailto:michael.d.kin...@intel.com>>; > devel@edk2.groups.io <mailto:devel@edk2.groups.io>; Sean Brogan > <spbro...@outlook.com <mailto:spbro...@outlook.com>>; ler...@redhat.com > <mailto:ler...@redhat.com> > Subject: Re: [edk2-devel] ci problems again > > On 6/12/20 6:12 PM, Kinney, Michael D wrote: > > Hi Ard, > > The CI agents are stateless, so it can not be stale > content. > > > So why is it looking for > ArmVirtPkg/PrePi/Scripts/PrePi- > > PIE.lds even > though not a single reference to it remains anywhere > in > > the tree? > > (grep'ing for PrePi-PIE.lds in the edk2 tree gives 0 > hits) > > > > Have you tried the pytools command that is being > used > > locally for this patch series? > > > No. Are there any instructions on how to use that? > > > > > > > -----Original Message----- > From: devel@edk2.groups.io <mailto:devel@edk2.groups.io> > <devel@edk2.groups.io <mailto:devel@edk2.groups.io>> > On > > Behalf Of Ard Biesheuvel > Sent: Friday, June 12, 2020 8:47 AM > To: Kinney, Michael D <michael.d.kin...@intel.com > <mailto:michael.d.kin...@intel.com>>; > devel@edk2.groups.io <mailto:devel@edk2.groups.io>; Sean Brogan > <spbro...@outlook.com <mailto:spbro...@outlook.com>>; ler...@redhat.com > <mailto:ler...@redhat.com> > Subject: Re: [edk2-devel] ci problems again > > On 6/12/20 5:36 PM, Kinney, Michael D wrote: > > Hi Ard, > > Please look at this more detailed log from Azure. > > https://dev.azure.com/tianocore/edk2- <https://dev.azure.com/tianocore/edk2-> > > > ci/_build/results?buildId=8170&view=logs&jobId=680b79ca- > > 2ce9-56d4-127b-ee6e95685750&j=680b79ca-2ce9-56d4- > 127b- > > ee6e95685750&t=1e12dbf9-b3ef-52bb-1019-4c2f443aef72 > > > It says it can not find some paths in ArmVirtPkg. > > > It is complaining about files that are being > removed > > by > > the series: > > Failed to get package for file > ArmVirtPkg/Include/Platform/Hidden.h. > Failed to get package for file > ArmVirtPkg/PrePi/Scripts/PrePi-PIE.lds. > > but I cannot find any remaining references to those > files in anywhere in > the tree, so it seems the CI has remembered the > existence of those files > in some way, and is now expecting them to be still > there. > > So I don't think there is any way I can fix this > myself. > > Someone needs > to wipe the build directory for these CI jobs. > > > > > > -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#61229): https://edk2.groups.io/g/devel/message/61229 Mute This Topic: https://groups.io/mt/74836016/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
