pushed as 71dab4b0eaa4214586b9e03c1c42c7562095353c Mike
> -----Original Message----- > From: Kinney, Michael D <michael.d.kin...@intel.com> > Sent: Tuesday, May 31, 2022 4:45 PM > To: devel@edk2.groups.io; Jayaprakash, N <n.jayaprak...@intel.com>; Kinney, > Michael D <michael.d.kin...@intel.com> > Subject: RE: [edk2-devel] [edk2-libc Patch 1/1] > AppPkg\Applications\Python\Python-3.6.8: GCC tool chain support > > Reviewed-by: Michael D Kinney <michael.d.kin...@intel.com> > > I do see that there are 3 documentation files in Python-3.6.8. > > One is from the Python project. The other 2 look like they both > contain EDK II build instructions. Perhaps these should be combined into > a single file for EDK II builds for all supported tool chains and archs. > > Mike > > > -----Original Message----- > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Jayaprakash, > > N > > Sent: Wednesday, May 18, 2022 11:42 AM > > To: devel@edk2.groups.io > > Subject: [edk2-devel] [edk2-libc Patch 1/1] > > AppPkg\Applications\Python\Python-3.6.8: GCC tool chain support > > > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3779 > > > > This commit contains the changes made to Python interpreter > > 3.6.8 to cross compile using the GCC tool chain on Ubuntu 20.04 LTS > > system for UEFI. The ctypes module has been excluded from GCC > > compilation as it requires > > additional porting work to use GCC tool chain support for UEFI > > environment. > > Added GCC Compilation steps as BKM document to facilitate the > > environment setup and > > compilation on Linux using GCC tool chain. Besides this provided a shell > > script to > > generate the usable Python UEFI package with all the dependencies taken > > from the > > build environment. > > > > Cc: Rebecca Cran <rebe...@nuviainc.com> > > Cc: Michael D Kinney <michael.d.kin...@intel.com> > > Signed-off-by: Jayaprakash N <n.jayaprak...@intel.com> > > --- > > .../Python-3.6.8/GCCCompilationBKMs.rst | 193 ++++++++++++++++++ > > .../PyMod-3.6.8/Include/pyconfig.h | 2 +- > > .../Python-3.6.8/PyMod-3.6.8/Modules/config.c | 7 +- > > .../PyMod-3.6.8/Modules/getpath.c | 2 +- > > .../Python/Python-3.6.8/Python368.inf | 21 +- > > .../Python-3.6.8/create_python368_pkg.sh | 100 +++++++++ > > 6 files changed, 312 insertions(+), 13 deletions(-) > > create mode 100644 > > AppPkg/Applications/Python/Python-3.6.8/GCCCompilationBKMs.rst > > create mode 100644 > > AppPkg/Applications/Python/Python-3.6.8/create_python368_pkg.sh > > > > diff --git a/AppPkg/Applications/Python/Python-3.6.8/GCCCompilationBKMs.rst > > b/AppPkg/Applications/Python/Python- > > 3.6.8/GCCCompilationBKMs.rst > > new file mode 100644 > > index 0000000..ed5fcbe > > --- /dev/null > > +++ b/AppPkg/Applications/Python/Python-3.6.8/GCCCompilationBKMs.rst > > @@ -0,0 +1,193 @@ > > +1. Introduction > > +--------------- > > +This is a Best-Known Methods document capturing the details of build > > environment setup for compiling the Python UEFI > > +interpreter version 3.6.8 using GCC tool chain on Linux based operating > > systems. These BKMs have been tested with WSL2 > > +(Windows Subsystem for Linux 2) based Ubuntu 20.04 LTS environment on > > Windows 10. Only 64-bit builds are possible due > > +to a limitation in the LibC support for GCC compiler tool chain on UEFI. > > If you are working on a Ubuntu 20.04 LTS system > > +you may skip the sections 1.1 and 1.2 > > + > > + > > +1.1. WSL2 Installation on Windows 10 OS > > +---------------------------------------- > > + > > +The WSL2 environment can be setup by following the instructions given in > > the below webpage > > +https://pureinfotech.com/install-windows-subsystem-linux-2-windows-10/ > > + > > +1.2. Installing Ubuntu 20.04 LTS on WSL2 on Windows 10 OS > > +---------------------------------------------------------- > > + > > +The instructions provided in the below webpage will help in installing the > > Ubuntu 20.04 LTS OS environment on WSL2 on Windows > 10 > > OS. > > +https://www.altisconsulting.com/au/insights/installing-ubuntu-bash-for-windows-10-wsl2-setup/ > > + > > +2. EDK2 build environment setup > > +------------------------------- > > + > > +You may follow the instructions provided in the below webpage to setup the > > edk2 build environment. > > + > > +https://github.com/tianocore/tianocore.github.io/wiki/Using-EDK-II-with-Native-GCC > > + > > +For the purposes of describing the BKMs, we will be using the following > > paths. > > +Edk2 source tree: $HOME/src/edk2 > > +Native GCC version >=4.x compiler installation: /usr/bin/gcc > > +Intel ASL Compiler installation: /usr/bin/iasl > > + > > + > > +2.1. Installing essential packages for GCC compiler > > +---------------------------------------------------- > > + > > +Install the below required essential packages to facilitate compilation of > > the edk2 repo using GCC compiler tool chain. > > +Several Ubuntu packages will be needed to set up the build environment for > > EDK II. > > + > > +The following command will install all required packages:: > > + > > + bash$ sudo apt-get update > > + bash$ sudo apt install build-essential > > + bash$ sudo apt install uuid-dev iasl git nasm python-is-python3 > > + > > +Package - Description > > + * build-essential : Informational list of build-essential packages > > + * uuid-dev : Universally Unique ID library (headers and static > > libraries) > > + * iasl : Intel ASL compiler/decompiler (also provided by acpica-tools) > > + * git : Support for git revision control system > > + * nasm : General-purpose x86 assembler > > + * python-is-python3 : Ubuntu 20.04 python command is 'python3' but > > edk2 tools use 'python' > > + > > +Once the above packages are installed, then follow the instructions in the > > webpage to setup the rest of the build > > +environment > > +https://github.com/tianocore/tianocore.github.io/wiki/Common-instructions > > + > > + > > +2.2. Cloning edk2 git repo > > +--------------------------- > > + > > +Get the edk2 source tree using Git commands. > > +Execute the below commands to clone the edk2 git repo:: > > + bash$ mkdir ~/src > > + bash$ cd ~/src > > + bash$ git clone https://github.com/tianocore/edk2 > > + > > +This will create a folder named edk2 as a clone of the edk2 git repo. > > +Execute the below commands to checkout / initialize all the git > > submodules:: > > + bash$ cd ~/src/edk2 > > + bash$ git submodule update --init > > + > > +The above command initializes all the submodules in the git repo > > +The below commands help to setup the environment variables required for > > building the edk2 repo:: > > + > > + bash$ make -C BaseTools > > + bash$ . edksetup.sh > > + > > +Run the below make command from ~/src folder i.e change working directory > > to ~/src and run the make command as given below :: > > + > > + bash$ cd ~/src > > + bash$ make -C edk2/BaseTools > > + > > +Setup build shell environment :: > > + > > + bash$ cd ~/src/edk2 > > + bash$ export EDK_TOOLS_PATH=$HOME/src/edk2/BaseTools > > + bash$ . edksetup.sh BaseTools > > + > > + > > +Set Build Target Information > > +For the Conf/target.txt file, find the following lines: > > +ACTIVE_PLATFORM = Nt32Pkg/Nt32Pkg.dsc > > + > > +TOOL_CHAIN_TAG = MYTOOLS > > + > > +And change the corresponding lines to match these: > > + > > +ACTIVE_PLATFORM = MdeModulePkg/MdeModulePkg.dsc > > + > > +TOOL_CHAIN_TAG = GCC5 > > + > > +Install GCC5 on Ubuntu 20.04 LTS by downloading it from > > +https://askubuntu.com/questions/1235819/ubuntu-20-04-gcc-version-lower-than-gcc-7 > > + > > +This requires latest version of the nasm tool to be installed. > > +The nasm latest version >= 2.15.05 > > + > > +Debian package for nasm 2.15.05 is available at > > + > > +https://ubuntu.pkgs.org/21.10/ubuntu-universe-amd64/nasm_2.15.05-1_amd64.deb.html > > + > > +Download it using the wget command as given below:: > > + > > + bash$ sudo wget > > http://archive.ubuntu.com/ubuntu/pool/universe/n/nasm/nasm_2.15.05-1_amd64.deb > > + > > + > > +Install the the above downloaded nasm debian package using the following > > command:: > > + > > + bash$ sudo dpkg -i nasm_2.15.05-1_amd64.deb > > + > > +2.3. Install additional packages required for building edk2 with GCC > > +--------------------------------------------------------------------- > > + > > +Install additional package required for building edk2 repo:: > > + > > + bash$ sudo apt install libx11-dev > > + bash$ sudo apt install libxext-dev > > + > > + > > +2.4 Clone edk2-libc and copy the contents to edk2 > > +-------------------------------------------------- > > + > > +Now clone the edk2-libc git repo by following / executing the below > > commands:: > > + > > + bash$ cd ~/src > > + bash$ git clone https://github.com/tianocore/edk2-libc.git > > + bash$ export PACKAGES_PATH=$HOME/src/edk2:$HOME/src/edk2-libc > > + > > + > > +Verify that you can build the hello world application from AppPkg under > > edk2-libc by running the following commands > > +Build hello.inf from edk2-libc applications:: > > + > > + $bash build -p AppPkg/AppPkg.dsc -m > > AppPkg/Applications/Hello/Hello.inf > > + > > +Copy the contents of edk2-libc to edk2 folder:: > > + > > + bash$ cp -rf ~/src/edk2-libc/* ~/src/edk2/ > > + > > +2.5. Build Python Interpreter using GCC > > +---------------------------------------- > > +Uncomment the line containing following inf file name in AppPkg/AppPkg.dsc > > file > > +AppPkg/Applications/Python/Python-3.6.8/Python368.inf > > + > > +Execute the below command to build the X64 version of the Python 3.6.8 > > interpreter using GCC tool chain:: > > + > > + bash$ cd AppPkg/Applications/Python/Python-3.6.8/ > > + bash$ python srcprep.py > > + bash$ cd ~/src/edk2 > > + bash$ build -a X64 -b RELEASE -p AppPkg/AppPkg.dsc -m > > AppPkg/Applications/Python/Python-3.6.8/Python368.inf > > + or > > + bash$ build -a X64 -b RELEASE -p AppPkg/AppPkg.dsc > > + > > + > > +2.5 Create Python UEFI package with all dependencies > > +----------------------------------------------------- > > + > > +To create an usable Python UEFI package with all the dependencies from the > > build environment, > > +you can use the bash shell script create_python368_pkg.sh available under > > /AppPkg/Applications/Python/Python-3.6.8 > > +folder. > > + > > +Use the following commands to create Python UEFI package:: > > + > > + bash$ cd ~/src/edk2/AppPkg/Applications/Python/Python-3.6.8/ > > + bash$ chmod 777 create_python368_pkg.sh > > + ./create_python368_pkg.sh GCC5 RELEASE X64 myPyUEFI > > + > > +where GCC5 refers to the tool chain, RELEASE refers to the build type, X64 > > refers to the architecture > > +myPyUEFI refers to the name of the folder where the Python UEFI package > > needs to be created. > > + > > +When you run the create_python368_pkg.sh without any parameters it lists > > the help > > +information as shown below: > > + > > +Shell Script to create Python EFI Package. > > + > > +Usage: ./create_python368_pkg.sh <ToolChain> <Target> <Architecture> > > <OutFolder> > > + > > +Where > > + * ToolChain : name of the tool chain such as GCC5 > > + * Target : build type such as RELEASE, DEBUG > > + * Architecture : Architecture such as X64 > > + * OutFolder : Output directory for creating the package > > diff --git > > a/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Include/pyconfig.h > > b/AppPkg/Applications/Python/Python- > > 3.6.8/PyMod-3.6.8/Include/pyconfig.h > > index d4685da..9d08982 100644 > > --- a/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Include/pyconfig.h > > +++ b/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Include/pyconfig.h > > @@ -870,7 +870,7 @@ > > #undef HAVE_THREAD_H > > > > /* Define to 1 if you have the 'timegm' function. */ > > -#undef HAVE_TIMEGM > > +#define HAVE_TIMEGM 1 > > > > /* Define to 1 if you have the 'times' function. */ > > #undef HAVE_TIMES > > diff --git > > a/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Modules/config.c > > b/AppPkg/Applications/Python/Python- > > 3.6.8/PyMod-3.6.8/Modules/config.c > > index 0e7f65e..7217a80 100644 > > --- a/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Modules/config.c > > +++ b/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Modules/config.c > > @@ -83,7 +83,10 @@ extern PyObject* PyInit__stat(void); > > extern PyObject* PyInit__opcode(void); > > extern PyObject* PyInit_faulthandler(void); > > // _ctypes > > +#if defined(UEFI_MSVC_32) || defined(UEFI_MSVC_64) > > extern PyObject* PyInit__ctypes(void); > > +#endif > > + > > extern PyObject* init_sqlite3(void); > > > > // EfiPy > > @@ -155,7 +158,9 @@ struct _inittab _PyImport_Inittab[] = { > > {"_string", PyInit__string}, > > {"_stat", PyInit__stat}, > > {"_opcode", PyInit__opcode}, > > - { "_ctypes", PyInit__ctypes }, > > +#if defined(UEFI_MSVC_32) || defined(UEFI_MSVC_64) > > + { "_ctypes", PyInit__ctypes }, > > +#endif > > /* Sentinel */ > > {0, 0} > > }; > > diff --git > > a/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Modules/getpath.c > > b/AppPkg/Applications/Python/Python- > > 3.6.8/PyMod-3.6.8/Modules/getpath.c > > index ad10784..7ee4d5e 100644 > > --- a/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Modules/getpath.c > > +++ b/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Modules/getpath.c > > @@ -112,7 +112,7 @@ > > #endif > > > > > > -#define SIFY_I( x ) L#x > > +#define SIFY_I( x ) L"" #x > > #define SIFY( y ) SIFY_I( y ) > > > > #ifndef PREFIX > > diff --git a/AppPkg/Applications/Python/Python-3.6.8/Python368.inf > > b/AppPkg/Applications/Python/Python-3.6.8/Python368.inf > > index 4ab0cd3..207b907 100644 > > --- a/AppPkg/Applications/Python/Python-3.6.8/Python368.inf > > +++ b/AppPkg/Applications/Python/Python-3.6.8/Python368.inf > > @@ -251,23 +251,24 @@ Modules/zlib > > Modules/zlib/zutil.c # > > > > #Modules/ctypes > > - PyMod-$(PYTHON_VERSION)/Modules/_ctypes/_ctypes.c # > > - Modules/_ctypes/stgdict.c # > > - Modules/_ctypes/libffi_msvc/prep_cif.c # > > - PyMod-$(PYTHON_VERSION)/Modules/_ctypes/malloc_closure.c # > > - PyMod-$(PYTHON_VERSION)/Modules/_ctypes/libffi_msvc/ffi.c # > > - Modules/_ctypes/cfield.c # > > - PyMod-$(PYTHON_VERSION)/Modules/_ctypes/callproc.c # > > - Modules/_ctypes/callbacks.c # > > + PyMod-$(PYTHON_VERSION)/Modules/_ctypes/_ctypes.c | MSFT # > > + Modules/_ctypes/stgdict.c | MSFT # > > + Modules/_ctypes/libffi_msvc/prep_cif.c | MSFT# > > + PyMod-$(PYTHON_VERSION)/Modules/_ctypes/malloc_closure.c | MSFT # > > + PyMod-$(PYTHON_VERSION)/Modules/_ctypes/libffi_msvc/ffi.c | MSFT # > > + Modules/_ctypes/cfield.c | MSFT # > > + PyMod-$(PYTHON_VERSION)/Modules/_ctypes/callproc.c | MSFT # > > + Modules/_ctypes/callbacks.c | MSFT # > > > > [Sources.IA32] > > - Modules/_ctypes/libffi_msvc/win32.c # > > + Modules/_ctypes/libffi_msvc/win32.c | MSFT # > > > > [Sources.X64] > > - Modules/_ctypes/libffi_msvc/win64.asm # > > + Modules/_ctypes/libffi_msvc/win64.asm | MSFT # > > > > [BuildOptions] > > MSFT:*_*_*_CC_FLAGS = /GL- /Oi- /wd4018 /wd4054 /wd4055 /wd4101 /wd4131 > > /wd4152 /wd4204 /wd4210 /wd4244 /wd4267 /wd4305 > > /wd4310 /wd4389 /wd4701 /wd4702 /wd4706 /wd4456 /wd4312 /wd4457 /wd4459 > > /wd4474 /wd4476 > > /I$(WORKSPACE)\AppPkg\Applications\Python\Python-3.6.8\Include > > /DHAVE_MEMMOVE /DUSE_PYEXPAT_CAPI /DXML_STATIC -D UEFI /WX- > > /DXML_POOR_ENTROPY /DUEFI_C_SOURCE > > + GCC:*_*_*_CC_FLAGS = -Wno-unused-function -Wno-format -Wno-error > > -fno-strict-aliasing - > > I$(WORKSPACE)/AppPkg/Applications/Python/Python-3.6.8/Include > > -DHAVE_MEMMOVE -DUSE_PYEXPAT_CAPI -DXML_STATIC -DXML_POOR_ENTROPY > > > > [BuildOptions.IA32] > > MSFT:*_*_*_CC_FLAGS = /DUEFI_MSVC_32 > > diff --git > > a/AppPkg/Applications/Python/Python-3.6.8/create_python368_pkg.sh > > b/AppPkg/Applications/Python/Python- > > 3.6.8/create_python368_pkg.sh > > new file mode 100644 > > index 0000000..f529fa4 > > --- /dev/null > > +++ b/AppPkg/Applications/Python/Python-3.6.8/create_python368_pkg.sh > > @@ -0,0 +1,100 @@ > > +#!/bin/bash > > + > > +TOOL_CHAIN_TAG=$1 > > +TARGET=$2 > > +ARCH=$3 > > +OUT_FOLDER=$4 > > + > > +function usage { > > + echo > > + echo Shell Script to create Python EFI Package. > > + echo > > + echo > > + echo "Usage: $0 <ToolChain> <Target> <Architecture> <OutFolder>" > > + echo "Where " > > + echo " ToolChain : name of the tool chain such as GCC5" > > + echo " Target : build type such as RELEASE, DEBUG" > > + echo " Architecture : Architecture such as X64" > > + echo " OutFolder : Output directory for creating the package" > > + echo > > +} > > + > > +function error { > > + echo Failed to Create Python EFI Package > > + echo Python368.efi application is not available at > > "Build/AppPkg/$TARGET"_"$TOOL_CHAIN_TAG/$ARCH/" > > + echo Follow the instructions in Py368ReadMe.txt to build Python > > interpreter > > + echo Then use this script to create a Python EFI package > > +} > > + > > + > > +# Validate the ToolChain input > > +if [ -z $TOOL_CHAIN_TAG ] > > +then > > + echo Invalid command line arguments passed, please see the below usage > > instructions > > + usage > > + exit > > +fi > > + > > +# Validate the Target input > > +if [ -z $TARGET ] > > +then > > + echo Invalid command line arguments passed, please see the below usage > > instructions > > + usage > > + exit > > +fi > > + > > +# Validate the Architecture input > > +if [ -z $ARCH ] > > +then > > + echo Invalid command line arguments passed, please see the below usage > > instructions > > + usage > > + exit > > +fi > > + > > +# Validate the OutFolder input > > +if [ -z $OUT_FOLDER ] > > +then > > + echo Invalid command line arguments passed, please see the below usage > > instructions > > + usage > > + exit > > +fi > > + > > +cd ../../../../ > > +PYTHON_BIN="Build/AppPkg/$TARGET"_"$TOOL_CHAIN_TAG/$ARCH/Python368.efi" > > +if [ ! -f $PYTHON_BIN ] > > +then > > + error > > + exit > > +fi > > + > > + > > +# create /EFI/Tools folder in the Output folder if it doesn't exists > > +if [ ! -d $OUT_FOLDER/EFI/Tools ] > > +then > > + mkdir -p $OUT_FOLDER/EFI/Tools > > +fi > > + > > +cp -f $PYTHON_BIN $OUT_FOLDER/EFI/Tools/ > > + > > +if [ ! -d $OUT_FOLDER/EFI/StdLib/lib/python36.8 ] > > +then > > + mkdir -p $OUT_FOLDER/EFI/StdLib/lib/python36.8 > > +fi > > + > > +if [ ! -d $OUT_FOLDER/EFI/StdLib/etc ] > > +then > > + mkdir -p $OUT_FOLDER/EFI/StdLib/etc > > +fi > > + > > +cp -rf AppPkg/Applications/Python/Python-3.6.8/Lib/* > > $OUT_FOLDER/EFI/StdLib/lib/python36.8/ > > +cp -rf StdLib/Efi/StdLib/etc/* $OUT_FOLDER/EFI/StdLib/etc/ > > + > > +if [ ${OUT_FOLDER:0:1} == "/" ] > > +then > > + echo Python EFI package available at $OUT_FOLDER > > +else > > + echo Python EFI package available at `pwd`/$OUT_FOLDER > > +fi > > + > > + > > + > > -- > > 2.33.0.windows.1 > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#90119): https://edk2.groups.io/g/devel/message/90119 Mute This Topic: https://groups.io/mt/91193074/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
