Author: Arvind Sudarsanam Date: 2024-10-31T09:39:55-05:00 New Revision: eeee5a44bbf24f2f30a313ecf66e6a76de365658
URL: https://github.com/llvm/llvm-project/commit/eeee5a44bbf24f2f30a313ecf66e6a76de365658 DIFF: https://github.com/llvm/llvm-project/commit/eeee5a44bbf24f2f30a313ecf66e6a76de365658.diff LOG: [Clang][SYCL] Introduce clang-sycl-linker to link SYCL offloading device code (Part 1 of many) (#112245) This PR is one of the many PRs in the SYCL upstreaming effort focusing on device code linking during the SYCL offload compilation process. RFC: https://discourse.llvm.org/t/rfc-offloading-design-for-sycl-offload-kind-and-spir-targets/74088 In this PR, we introduce a new tool that will be used to perform device code linking for SYCL offload kind. It accepts SYCL device objects in LLVM IR bitcode format and will generate a fully linked device object that can then be wrapped and linked into the host object. A primary use case for this tool is to perform device code linking for objects with SYCL offload kind inside the clang-linker-wrapper. It can also be invoked via clang driver as follows: `clang --target=spirv64 --sycl-link input.bc` Device code linking for SYCL offloading kind has a number of known quirks that makes it difficult to use in a unified offloading setting. Two of the primary issues are: 1. Several finalization steps are required to be run on the fully-linked LLVM IR bitcode to gaurantee conformance to SYCL standards. This step is unique to SYCL offloading compilation flow. 2. SPIR-V LLVM Translator tool is an extenal tool and hence SPIR-V IR code generation cannot be done as part of LTO. This limitation will be lifted once SPIR-V backend is available as a viable LLVM backend. Hence, we introduce this new tool to provide a clean wrapper to perform SYCL device linking. Co-Author: Michael Toguchi Thanks --------- Signed-off-by: Arvind Sudarsanam <arvind.sudarsa...@intel.com> Added: clang/docs/ClangSYCLLinker.rst clang/test/Driver/clang-sycl-linker-test.cpp clang/test/Driver/sycl-link-spirv-target.cpp clang/tools/clang-sycl-linker/CMakeLists.txt clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp clang/tools/clang-sycl-linker/SYCLLinkOpts.td Modified: clang/docs/index.rst clang/include/clang/Driver/Options.td clang/lib/Driver/Driver.cpp clang/lib/Driver/ToolChains/SPIRV.cpp clang/lib/Driver/ToolChains/SPIRV.h clang/test/CMakeLists.txt clang/test/lit.cfg.py clang/tools/CMakeLists.txt Removed: ################################################################################ diff --git a/clang/docs/ClangSYCLLinker.rst b/clang/docs/ClangSYCLLinker.rst new file mode 100644 index 00000000000000..c1a794a2f65f64 --- /dev/null +++ b/clang/docs/ClangSYCLLinker.rst @@ -0,0 +1,82 @@ +======================= +Clang SYCL Linker +======================= + +.. contents:: + :local: + +.. _clang-sycl-linker: + +Introduction +============ + +This tool works as a wrapper around the SYCL device code linking process. +The purpose of this tool is to provide an interface to link SYCL device bitcode +in LLVM IR format, SYCL device bitcode in SPIR-V IR format, and native binary +objects, and then use the SPIR-V LLVM Translator tool on fully linked device +objects to produce the final output. +After the linking stage, the fully linked device code in LLVM IR format may +undergo several SYCL-specific finalization steps before the SPIR-V code +generation step. +The tool will also support the Ahead-Of-Time (AOT) compilation flow. AOT +compilation is the process of invoking the back-end at compile time to produce +the final binary, as opposed to just-in-time (JIT) compilation when final code +generation is deferred until application runtime. + +Device code linking for SYCL offloading has several known quirks that +make it diff icult to use in a unified offloading setting. Two of the primary +issues are: +1. Several finalization steps are required to be run on the fully linked LLVM +IR bitcode to guarantee conformance to SYCL standards. This step is unique to +the SYCL offloading compilation flow. +2. The SPIR-V LLVM Translator tool is an external tool and hence SPIR-V IR code +generation cannot be done as part of LTO. This limitation can be lifted once +the SPIR-V backend is available as a viable LLVM backend. + +This tool has been proposed to work around these issues. + +Usage +===== + +This tool can be used with the following options. Several of these options will +be passed down to downstream tools like 'llvm-link', 'llvm-spirv', etc. + +.. code-block:: console + + OVERVIEW: A utility that wraps around the SYCL device code linking process. + This enables linking and code generation for SPIR-V JIT targets and AOT + targets. + + USAGE: clang-sycl-linker [options] + + OPTIONS: + --arch <value> Specify the name of the target architecture. + --dry-run Print generated commands without running. + -g Specify that this was a debug compile. + -help-hidden Display all available options + -help Display available options (--help-hidden for more) + --library-path=<dir> Set the library path for SYCL device libraries + --device-libs=<value> A comma separated list of device libraries that are linked during the device link + -o <path> Path to file to write output + --save-temps Save intermediate results + --triple <value> Specify the target triple. + --version Display the version number and exit + -v Print verbose information + -spirv-dump-device-code=<dir> Directory to dump SPIR-V IR code into + -is-windows-msvc-env Specify if we are compiling under windows environment + -llvm-spirv-options=<value> Pass options to llvm-spirv tool + --llvm-spirv-path=<dir> Set the system llvm-spirv path + +Example +======= + +This tool is intended to be invoked when targeting any of the target offloading +toolchains. When the --sycl-link option is passed to the clang driver, the +driver will invoke the linking job of the target offloading toolchain, which in +turn will invoke this tool. This tool can be used to create one or more fully +linked device images that are ready to be wrapped and linked with host code to +generate the final executable. + +.. code-block:: console + + clang-sycl-linker --triple spirv64 --arch native input.bc diff --git a/clang/docs/index.rst b/clang/docs/index.rst index 66a4540a0bcacf..3c473f93e5224a 100644 --- a/clang/docs/index.rst +++ b/clang/docs/index.rst @@ -98,6 +98,7 @@ Using Clang Tools ClangOffloadBundler ClangOffloadPackager ClangRepl + ClangSYCLLinker Design Documents ================ diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td index 5115b3d3e09c7c..c8bc2fe377b8ec 100644 --- a/clang/include/clang/Driver/Options.td +++ b/clang/include/clang/Driver/Options.td @@ -6775,7 +6775,10 @@ def fsycl : Flag<["-"], "fsycl">, def fno_sycl : Flag<["-"], "fno-sycl">, Visibility<[ClangOption, CLOption]>, Group<sycl_Group>, HelpText<"Disables SYCL kernels compilation for device">; - +def sycl_link : Flag<["--"], "sycl-link">, Flags<[HelpHidden]>, + Visibility<[ClangOption, CLOption]>, + Group<sycl_Group>, HelpText<"Perform link through clang-sycl-linker via the target " + "offloading toolchain.">; // OS-specific options let Flags = [TargetSpecific] in { defm android_pad_segment : BooleanFFlag<"android-pad-segment">, Group<f_Group>; diff --git a/clang/lib/Driver/Driver.cpp b/clang/lib/Driver/Driver.cpp index 9878a9dad78d40..083035dee43028 100644 --- a/clang/lib/Driver/Driver.cpp +++ b/clang/lib/Driver/Driver.cpp @@ -4791,6 +4791,11 @@ Action *Driver::ConstructPhaseAction( if (Phase == phases::Assemble && Input->getType() != types::TY_PP_Asm) return Input; + // Use of --sycl-link will only allow for the link phase to occur. This is + // for all input files. + if (Args.hasArg(options::OPT_sycl_link) && Phase != phases::Link) + return Input; + // Build the appropriate action. switch (Phase) { case phases::Link: diff --git a/clang/lib/Driver/ToolChains/SPIRV.cpp b/clang/lib/Driver/ToolChains/SPIRV.cpp index ce900600cbee51..659da5c7f25aa9 100644 --- a/clang/lib/Driver/ToolChains/SPIRV.cpp +++ b/clang/lib/Driver/ToolChains/SPIRV.cpp @@ -95,7 +95,21 @@ void SPIRV::Linker::ConstructJob(Compilation &C, const JobAction &JA, CmdArgs.push_back("-o"); CmdArgs.push_back(Output.getFilename()); + // Use of --sycl-link will call the clang-sycl-linker instead of + // the default linker (spirv-link). + if (Args.hasArg(options::OPT_sycl_link)) + Linker = ToolChain.GetProgramPath("clang-sycl-linker"); C.addCommand(std::make_unique<Command>(JA, *this, ResponseFileSupport::None(), Args.MakeArgString(Linker), CmdArgs, Inputs, Output)); } + +SPIRVToolChain::SPIRVToolChain(const Driver &D, const llvm::Triple &Triple, + const ArgList &Args) + : ToolChain(D, Triple, Args) { + // TODO: Revisit need/use of --sycl-link option once SYCL toolchain is + // available and SYCL linking support is moved there. + NativeLLVMSupport = Args.hasArg(options::OPT_sycl_link); +} + +bool SPIRVToolChain::HasNativeLLVMSupport() const { return NativeLLVMSupport; } diff --git a/clang/lib/Driver/ToolChains/SPIRV.h b/clang/lib/Driver/ToolChains/SPIRV.h index d4247ee0557f4b..d59a8c76ed4737 100644 --- a/clang/lib/Driver/ToolChains/SPIRV.h +++ b/clang/lib/Driver/ToolChains/SPIRV.h @@ -57,8 +57,7 @@ class LLVM_LIBRARY_VISIBILITY SPIRVToolChain final : public ToolChain { public: SPIRVToolChain(const Driver &D, const llvm::Triple &Triple, - const llvm::opt::ArgList &Args) - : ToolChain(D, Triple, Args) {} + const llvm::opt::ArgList &Args); bool useIntegratedAs() const override { return true; } @@ -72,6 +71,7 @@ class LLVM_LIBRARY_VISIBILITY SPIRVToolChain final : public ToolChain { } bool isPICDefaultForced() const override { return false; } bool SupportsProfiling() const override { return false; } + bool HasNativeLLVMSupport() const override; clang::driver::Tool *SelectTool(const JobAction &JA) const override; @@ -81,6 +81,7 @@ class LLVM_LIBRARY_VISIBILITY SPIRVToolChain final : public ToolChain { private: clang::driver::Tool *getTranslator() const; + bool NativeLLVMSupport; }; } // namespace toolchains diff --git a/clang/test/CMakeLists.txt b/clang/test/CMakeLists.txt index 98829d53db934f..5369dc92f69e8a 100644 --- a/clang/test/CMakeLists.txt +++ b/clang/test/CMakeLists.txt @@ -80,6 +80,7 @@ list(APPEND CLANG_TEST_DEPS clang-nvlink-wrapper clang-offload-bundler clang-offload-packager + clang-sycl-linker diagtool hmaptool ) diff --git a/clang/test/Driver/clang-sycl-linker-test.cpp b/clang/test/Driver/clang-sycl-linker-test.cpp new file mode 100644 index 00000000000000..f358900b4fbd81 --- /dev/null +++ b/clang/test/Driver/clang-sycl-linker-test.cpp @@ -0,0 +1,48 @@ +// Tests the clang-sycl-linker tool. +// +// Test a simple case without arguments. +// RUN: %clangxx -emit-llvm -c %s -o %t_1.bc +// RUN: %clangxx -emit-llvm -c %s -o %t_2.bc +// RUN: clang-sycl-linker --dry-run -triple spirv64 %t_1.bc %t_2.bc -o a.spv 2>&1 \ +// RUN: | FileCheck %s --check-prefix=SIMPLE +// SIMPLE: "{{.*}}llvm-link{{.*}}" {{.*}}.bc {{.*}}.bc -o [[FIRSTLLVMLINKOUT:.*]].bc --suppress-warnings +// SIMPLE-NEXT: "{{.*}}llvm-spirv{{.*}}" {{.*}}-o a.spv [[FIRSTLLVMLINKOUT]].bc +// +// Test that llvm-link is not called when only one input is present. +// RUN: clang-sycl-linker --dry-run -triple spirv64 %t_1.bc -o a.spv 2>&1 \ +// RUN: | FileCheck %s --check-prefix=SIMPLE-NO-LINK +// SIMPLE-NO-LINK: "{{.*}}llvm-spirv{{.*}}" {{.*}}-o a.spv {{.*}}.bc +// +// Test a simple case with device library files specified. +// RUN: touch %T/lib1.bc +// RUN: touch %T/lib2.bc +// RUN: clang-sycl-linker --dry-run -triple spirv64 %t_1.bc %t_2.bc --library-path=%T --device-libs=lib1.bc,lib2.bc -o a.spv 2>&1 \ +// RUN: | FileCheck %s --check-prefix=DEVLIBS +// DEVLIBS: "{{.*}}llvm-link{{.*}}" {{.*}}.bc {{.*}}.bc -o [[FIRSTLLVMLINKOUT:.*]].bc --suppress-warnings +// DEVLIBS-NEXT: "{{.*}}llvm-link{{.*}}" -only-needed [[FIRSTLLVMLINKOUT]].bc {{.*}}lib1.bc {{.*}}lib2.bc -o [[SECONDLLVMLINKOUT:.*]].bc --suppress-warnings +// DEVLIBS-NEXT: "{{.*}}llvm-spirv{{.*}}" {{.*}}-o a.spv [[SECONDLLVMLINKOUT]].bc +// +// Test a simple case with .o (fat object) as input. +// TODO: Remove this test once fat object support is added. +// RUN: %clangxx -c %s -o %t.o +// RUN: not clang-sycl-linker --dry-run -triple spirv64 %t.o -o a.spv 2>&1 \ +// RUN: | FileCheck %s --check-prefix=FILETYPEERROR +// FILETYPEERROR: Unsupported file type +// +// Test to see if device library related errors are emitted. +// RUN: not clang-sycl-linker --dry-run -triple spirv64 %t_1.bc %t_2.bc --library-path=%T --device-libs= -o a.spv 2>&1 \ +// RUN: | FileCheck %s --check-prefix=DEVLIBSERR1 +// DEVLIBSERR1: Number of device library files cannot be zero +// RUN: not clang-sycl-linker --dry-run -triple spirv64 %t_1.bc %t_2.bc --library-path=%T --device-libs=lib1.bc,lib2.bc,lib3.bc -o a.spv 2>&1 \ +// RUN: | FileCheck %s --check-prefix=DEVLIBSERR2 +// DEVLIBSERR2: '{{.*}}lib3.bc' SYCL device library file is not found +// +// Test if correct set of llvm-spirv options are emitted for windows environment. +// RUN: clang-sycl-linker --dry-run -triple spirv64 --is-windows-msvc-env %t_1.bc %t_2.bc -o a.spv 2>&1 \ +// RUN: | FileCheck %s --check-prefix=LLVMOPTSWIN +// LLVMOPTSWIN: -spirv-debug-info-version=ocl-100 -spirv-allow-extra-diexpressions -spirv-allow-unknown-intrinsics=llvm.genx. -spirv-ext= +// +// Test if correct set of llvm-spirv options are emitted for linux environment. +// RUN: clang-sycl-linker --dry-run -triple spirv64 %t_1.bc %t_2.bc -o a.spv 2>&1 \ +// RUN: | FileCheck %s --check-prefix=LLVMOPTSLIN +// LLVMOPTSLIN: -spirv-debug-info-version=nonsemantic-shader-200 -spirv-allow-unknown-intrinsics=llvm.genx. -spirv-ext= diff --git a/clang/test/Driver/sycl-link-spirv-target.cpp b/clang/test/Driver/sycl-link-spirv-target.cpp new file mode 100644 index 00000000000000..85566c67ea92b0 --- /dev/null +++ b/clang/test/Driver/sycl-link-spirv-target.cpp @@ -0,0 +1,9 @@ +// Tests the driver when linking LLVM IR bitcode files and targeting SPIR-V +// architecture. +// +// Test that -Xlinker options are being passed to clang-sycl-linker. +// RUN: touch %t.bc +// RUN: %clangxx -### --target=spirv64 --sycl-link -Xlinker --llvm-spirv-path=/tmp \ +// RUN: -Xlinker --library-path=/tmp -Xlinker --device-libs=lib1.bc,lib2.bc %t.bc 2>&1 \ +// RUN: | FileCheck %s -check-prefix=XLINKEROPTS +// XLINKEROPTS: "{{.*}}clang-sycl-linker{{.*}}" "--llvm-spirv-path=/tmp" "--library-path=/tmp" "--device-libs=lib1.bc,lib2.bc" "{{.*}}.bc" "-o" "a.out" diff --git a/clang/test/lit.cfg.py b/clang/test/lit.cfg.py index 92a3361ce672e2..4d3469aba4bb8d 100644 --- a/clang/test/lit.cfg.py +++ b/clang/test/lit.cfg.py @@ -96,6 +96,7 @@ "yaml2obj", "clang-linker-wrapper", "clang-nvlink-wrapper", + "clang-sycl-linker", "llvm-lto", "llvm-lto2", "llvm-profdata", diff --git a/clang/tools/CMakeLists.txt b/clang/tools/CMakeLists.txt index 88e29412e54350..98c018e96848df 100644 --- a/clang/tools/CMakeLists.txt +++ b/clang/tools/CMakeLists.txt @@ -12,6 +12,7 @@ add_clang_subdirectory(clang-nvlink-wrapper) add_clang_subdirectory(clang-offload-packager) add_clang_subdirectory(clang-offload-bundler) add_clang_subdirectory(clang-scan-deps) +add_clang_subdirectory(clang-sycl-linker) add_clang_subdirectory(clang-installapi) if(HAVE_CLANG_REPL_SUPPORT) add_clang_subdirectory(clang-repl) diff --git a/clang/tools/clang-sycl-linker/CMakeLists.txt b/clang/tools/clang-sycl-linker/CMakeLists.txt new file mode 100644 index 00000000000000..e86ec2736d3c03 --- /dev/null +++ b/clang/tools/clang-sycl-linker/CMakeLists.txt @@ -0,0 +1,28 @@ +set(LLVM_LINK_COMPONENTS + ${LLVM_TARGETS_TO_BUILD} + Option + ) + +set(LLVM_TARGET_DEFINITIONS SYCLLinkOpts.td) +tablegen(LLVM SYCLLinkOpts.inc -gen-opt-parser-defs) +add_public_tablegen_target(SYCLLinkerOpts) + +if(NOT CLANG_BUILT_STANDALONE) + set(tablegen_deps intrinsics_gen SYCLLinkerOpts) +endif() + +add_clang_tool(clang-sycl-linker + ClangSYCLLinker.cpp + + DEPENDS + ${tablegen_deps} + ) + +set(CLANG_SYCL_LINKER_LIB_DEPS + clangBasic + ) + +target_link_libraries(clang-sycl-linker + PRIVATE + ${CLANG_SYCL_LINKER_LIB_DEPS} + ) diff --git a/clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp b/clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp new file mode 100644 index 00000000000000..0639b95c76e218 --- /dev/null +++ b/clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp @@ -0,0 +1,506 @@ +//=-------- clang-sycl-linker/ClangSYCLLinker.cpp - SYCL Linker util -------=// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===---------------------------------------------------------------------===// +// +// This tool executes a sequence of steps required to link device code in SYCL +// device images. SYCL device code linking requires a complex sequence of steps +// that include linking of llvm bitcode files, linking device library files +// with the fully linked source bitcode file(s), running several SYCL specific +// post-link steps on the fully linked bitcode file(s), and finally generating +// target-specific device code. +//===---------------------------------------------------------------------===// + +#include "clang/Basic/Version.h" + +#include "llvm/ADT/StringExtras.h" +#include "llvm/BinaryFormat/Magic.h" +#include "llvm/Bitcode/BitcodeWriter.h" +#include "llvm/CodeGen/CommandFlags.h" +#include "llvm/IR/DiagnosticPrinter.h" +#include "llvm/IRReader/IRReader.h" +#include "llvm/LTO/LTO.h" +#include "llvm/Object/Archive.h" +#include "llvm/Object/ArchiveWriter.h" +#include "llvm/Object/Binary.h" +#include "llvm/Object/ELFObjectFile.h" +#include "llvm/Object/IRObjectFile.h" +#include "llvm/Object/ObjectFile.h" +#include "llvm/Object/OffloadBinary.h" +#include "llvm/Option/ArgList.h" +#include "llvm/Option/OptTable.h" +#include "llvm/Option/Option.h" +#include "llvm/Remarks/HotnessThresholdParser.h" +#include "llvm/Support/CommandLine.h" +#include "llvm/Support/FileOutputBuffer.h" +#include "llvm/Support/FileSystem.h" +#include "llvm/Support/InitLLVM.h" +#include "llvm/Support/MemoryBuffer.h" +#include "llvm/Support/Path.h" +#include "llvm/Support/Program.h" +#include "llvm/Support/Signals.h" +#include "llvm/Support/StringSaver.h" +#include "llvm/Support/TargetSelect.h" +#include "llvm/Support/TimeProfiler.h" +#include "llvm/Support/WithColor.h" + +using namespace llvm; +using namespace llvm::opt; +using namespace llvm::object; + +/// Save intermediary results. +static bool SaveTemps = false; + +/// Print arguments without executing. +static bool DryRun = false; + +/// Print verbose output. +static bool Verbose = false; + +/// Filename of the output being created. +static StringRef OutputFile; + +/// Directory to dump SPIR-V IR if requested by user. +static SmallString<128> SPIRVDumpDir; + +static void printVersion(raw_ostream &OS) { + OS << clang::getClangToolFullVersion("clang-sycl-linker") << '\n'; +} + +/// The value of `argv[0]` when run. +static const char *Executable; + +/// Temporary files to be cleaned up. +static SmallVector<SmallString<128>> TempFiles; + +namespace { +// Must not overlap with llvm::opt::DriverFlag. +enum LinkerFlags { LinkerOnlyOption = (1 << 4) }; + +enum ID { + OPT_INVALID = 0, // This is not an option ID. +#define OPTION(...) LLVM_MAKE_OPT_ID(__VA_ARGS__), +#include "SYCLLinkOpts.inc" + LastOption +#undef OPTION +}; + +#define PREFIX(NAME, VALUE) \ + static constexpr StringLiteral NAME##_init[] = VALUE; \ + static constexpr ArrayRef<StringLiteral> NAME(NAME##_init, \ + std::size(NAME##_init) - 1); +#include "SYCLLinkOpts.inc" +#undef PREFIX + +static constexpr OptTable::Info InfoTable[] = { +#define OPTION(...) LLVM_CONSTRUCT_OPT_INFO(__VA_ARGS__), +#include "SYCLLinkOpts.inc" +#undef OPTION +}; + +class LinkerOptTable : public opt::GenericOptTable { +public: + LinkerOptTable() : opt::GenericOptTable(InfoTable) {} +}; + +const OptTable &getOptTable() { + static const LinkerOptTable *Table = []() { + auto Result = std::make_unique<LinkerOptTable>(); + return Result.release(); + }(); + return *Table; +} + +[[noreturn]] void reportError(Error E) { + outs().flush(); + logAllUnhandledErrors(std::move(E), WithColor::error(errs(), Executable)); + exit(EXIT_FAILURE); +} + +std::string getMainExecutable(const char *Name) { + void *Ptr = (void *)(intptr_t)&getMainExecutable; + auto COWPath = sys::fs::getMainExecutable(Name, Ptr); + return sys::path::parent_path(COWPath).str(); +} + +Expected<StringRef> createTempFile(const ArgList &Args, const Twine &Prefix, + StringRef Extension) { + SmallString<128> OutputFile; + if (Args.hasArg(OPT_save_temps)) { + // Generate a unique path name without creating a file + sys::fs::createUniquePath(Prefix + "-%%%%%%." + Extension, OutputFile, + /*MakeAbsolute=*/false); + } else { + if (std::error_code EC = + sys::fs::createTemporaryFile(Prefix, Extension, OutputFile)) + return createFileError(OutputFile, EC); + } + + TempFiles.emplace_back(std::move(OutputFile)); + return TempFiles.back(); +} + +Expected<std::string> findProgram(const ArgList &Args, StringRef Name, + ArrayRef<StringRef> Paths) { + if (Args.hasArg(OPT_dry_run)) + return Name.str(); + ErrorOr<std::string> Path = sys::findProgramByName(Name, Paths); + if (!Path) + Path = sys::findProgramByName(Name); + if (!Path) + return createStringError(Path.getError(), + "Unable to find '" + Name + "' in path"); + return *Path; +} + +void printCommands(ArrayRef<StringRef> CmdArgs) { + if (CmdArgs.empty()) + return; + + llvm::errs() << " \"" << CmdArgs.front() << "\" "; + llvm::errs() << llvm::join(std::next(CmdArgs.begin()), CmdArgs.end(), " ") + << "\n"; +} + +/// Execute the command \p ExecutablePath with the arguments \p Args. +Error executeCommands(StringRef ExecutablePath, ArrayRef<StringRef> Args) { + if (Verbose || DryRun) + printCommands(Args); + + if (!DryRun) + if (sys::ExecuteAndWait(ExecutablePath, Args)) + return createStringError( + "'%s' failed", sys::path::filename(ExecutablePath).str().c_str()); + return Error::success(); +} + +Expected<SmallVector<std::string>> getInput(const ArgList &Args) { + // Collect all input bitcode files to be passed to llvm-link. + SmallVector<std::string> BitcodeFiles; + for (const opt::Arg *Arg : Args.filtered(OPT_INPUT)) { + std::optional<std::string> Filename = std::string(Arg->getValue()); + if (!Filename || !sys::fs::exists(*Filename) || + sys::fs::is_directory(*Filename)) + continue; + file_magic Magic; + if (auto EC = identify_magic(*Filename, Magic)) + return createStringError("Failed to open file " + *Filename); + // TODO: Current use case involves LLVM IR bitcode files as input. + // This will be extended to support objects and SPIR-V IR files. + if (Magic != file_magic::bitcode) + return createStringError("Unsupported file type"); + BitcodeFiles.push_back(*Filename); + } + return BitcodeFiles; +} + +/// Link all SYCL device input files into one before adding device library +/// files. Device linking is performed using llvm-link tool. +/// 'InputFiles' is the list of all LLVM IR device input files. +/// 'Args' encompasses all arguments required for linking device code and will +/// be parsed to generate options required to be passed into llvm-link. +Expected<StringRef> linkDeviceInputFiles(ArrayRef<std::string> InputFiles, + const ArgList &Args) { + llvm::TimeTraceScope TimeScope("SYCL LinkDeviceInputFiles"); + + assert(InputFiles.size() && "No inputs to llvm-link"); + // Early check to see if there is only one input. + if (InputFiles.size() < 2) + return InputFiles[0]; + + Expected<std::string> LLVMLinkPath = + findProgram(Args, "llvm-link", {getMainExecutable("llvm-link")}); + if (!LLVMLinkPath) + return LLVMLinkPath.takeError(); + + SmallVector<StringRef> CmdArgs; + CmdArgs.push_back(*LLVMLinkPath); + for (auto &File : InputFiles) + CmdArgs.push_back(File); + // Create a new file to write the linked device file to. + auto OutFileOrErr = + createTempFile(Args, sys::path::filename(OutputFile), "bc"); + if (!OutFileOrErr) + return OutFileOrErr.takeError(); + CmdArgs.push_back("-o"); + CmdArgs.push_back(*OutFileOrErr); + CmdArgs.push_back("--suppress-warnings"); + if (Error Err = executeCommands(*LLVMLinkPath, CmdArgs)) + return std::move(Err); + return *OutFileOrErr; +} + +// This utility function is used to gather all SYCL device library files that +// will be linked with input device files. +// The list of files and its location are passed from driver. +Expected<SmallVector<std::string>> getSYCLDeviceLibs(const ArgList &Args) { + SmallVector<std::string> DeviceLibFiles; + StringRef LibraryPath; + if (Arg *A = Args.getLastArg(OPT_library_path_EQ)) + LibraryPath = A->getValue(); + if (LibraryPath.empty()) + return DeviceLibFiles; + if (Arg *A = Args.getLastArg(OPT_device_libs_EQ)) { + if (A->getValues().size() == 0) + return createStringError( + inconvertibleErrorCode(), + "Number of device library files cannot be zero."); + for (StringRef Val : A->getValues()) { + SmallString<128> LibName(LibraryPath); + llvm::sys::path::append(LibName, Val); + if (llvm::sys::fs::exists(LibName)) + DeviceLibFiles.push_back(std::string(LibName)); + else + return createStringError(inconvertibleErrorCode(), + "\'" + std::string(LibName) + "\'" + + " SYCL device library file is not found."); + } + } + return DeviceLibFiles; +} + +/// Link all device library files and input file into one LLVM IR file. This +/// linking is performed using llvm-link tool. +/// 'InputFiles' is the list of all LLVM IR device input files. +/// 'Args' encompasses all arguments required for linking device code and will +/// be parsed to generate options required to be passed into llvm-link tool. +static Expected<StringRef> linkDeviceLibFiles(StringRef InputFile, + const ArgList &Args) { + llvm::TimeTraceScope TimeScope("LinkDeviceLibraryFiles"); + + auto SYCLDeviceLibFiles = getSYCLDeviceLibs(Args); + if (!SYCLDeviceLibFiles) + return SYCLDeviceLibFiles.takeError(); + if ((*SYCLDeviceLibFiles).empty()) + return InputFile; + + Expected<std::string> LLVMLinkPath = + findProgram(Args, "llvm-link", {getMainExecutable("llvm-link")}); + if (!LLVMLinkPath) + return LLVMLinkPath.takeError(); + + // Create a new file to write the linked device file to. + auto OutFileOrErr = + createTempFile(Args, sys::path::filename(OutputFile), "bc"); + if (!OutFileOrErr) + return OutFileOrErr.takeError(); + + SmallVector<StringRef, 8> CmdArgs; + CmdArgs.push_back(*LLVMLinkPath); + CmdArgs.push_back("-only-needed"); + CmdArgs.push_back(InputFile); + for (auto &File : *SYCLDeviceLibFiles) + CmdArgs.push_back(File); + CmdArgs.push_back("-o"); + CmdArgs.push_back(*OutFileOrErr); + CmdArgs.push_back("--suppress-warnings"); + if (Error Err = executeCommands(*LLVMLinkPath, CmdArgs)) + return std::move(Err); + return *OutFileOrErr; +} + +/// Add any llvm-spirv option that relies on a specific Triple in addition +/// to user supplied options. +static void getSPIRVTransOpts(const ArgList &Args, + SmallVector<StringRef, 8> &TranslatorArgs, + const llvm::Triple Triple) { + // Enable NonSemanticShaderDebugInfo.200 for non-Windows + const bool IsWindowsMSVC = + Triple.isWindowsMSVCEnvironment() || Args.hasArg(OPT_is_windows_msvc_env); + const bool EnableNonSemanticDebug = !IsWindowsMSVC; + if (EnableNonSemanticDebug) { + TranslatorArgs.push_back( + "-spirv-debug-info-version=nonsemantic-shader-200"); + } else { + TranslatorArgs.push_back("-spirv-debug-info-version=ocl-100"); + // Prevent crash in the translator if input IR contains DIExpression + // operations which don't have mapping to OpenCL.DebugInfo.100 spec. + TranslatorArgs.push_back("-spirv-allow-extra-diexpressions"); + } + std::string UnknownIntrinsics("-spirv-allow-unknown-intrinsics=llvm.genx."); + + TranslatorArgs.push_back(Args.MakeArgString(UnknownIntrinsics)); + + // Disable all the extensions by default + std::string ExtArg("-spirv-ext=-all"); + std::string DefaultExtArg = + ",+SPV_EXT_shader_atomic_float_add,+SPV_EXT_shader_atomic_float_min_max" + ",+SPV_KHR_no_integer_wrap_decoration,+SPV_KHR_float_controls" + ",+SPV_KHR_expect_assume,+SPV_KHR_linkonce_odr"; + std::string INTELExtArg = + ",+SPV_INTEL_subgroups,+SPV_INTEL_media_block_io" + ",+SPV_INTEL_device_side_avc_motion_estimation" + ",+SPV_INTEL_fpga_loop_controls,+SPV_INTEL_unstructured_loop_controls" + ",+SPV_INTEL_fpga_reg,+SPV_INTEL_blocking_pipes" + ",+SPV_INTEL_function_pointers,+SPV_INTEL_kernel_attributes" + ",+SPV_INTEL_io_pipes,+SPV_INTEL_inline_assembly" + ",+SPV_INTEL_arbitrary_precision_integers" + ",+SPV_INTEL_float_controls2,+SPV_INTEL_vector_compute" + ",+SPV_INTEL_fast_composite" + ",+SPV_INTEL_arbitrary_precision_fixed_point" + ",+SPV_INTEL_arbitrary_precision_floating_point" + ",+SPV_INTEL_variable_length_array,+SPV_INTEL_fp_fast_math_mode" + ",+SPV_INTEL_long_constant_composite" + ",+SPV_INTEL_arithmetic_fence" + ",+SPV_INTEL_global_variable_decorations" + ",+SPV_INTEL_cache_controls" + ",+SPV_INTEL_fpga_buffer_location" + ",+SPV_INTEL_fpga_argument_interfaces" + ",+SPV_INTEL_fpga_invocation_pipelining_attributes" + ",+SPV_INTEL_fpga_latency_control" + ",+SPV_INTEL_task_sequence" + ",+SPV_KHR_shader_clock" + ",+SPV_INTEL_bindless_images"; + ExtArg = ExtArg + DefaultExtArg + INTELExtArg; + ExtArg += ",+SPV_INTEL_token_type" + ",+SPV_INTEL_bfloat16_conversion" + ",+SPV_INTEL_joint_matrix" + ",+SPV_INTEL_hw_thread_queries" + ",+SPV_KHR_uniform_group_instructions" + ",+SPV_INTEL_masked_gather_scatter" + ",+SPV_INTEL_tensor_float32_conversion" + ",+SPV_INTEL_optnone" + ",+SPV_KHR_non_semantic_info" + ",+SPV_KHR_cooperative_matrix"; + TranslatorArgs.push_back(Args.MakeArgString(ExtArg)); +} + +/// Run LLVM to SPIR-V translation. +/// Converts 'File' from LLVM bitcode to SPIR-V format using llvm-spirv tool. +/// 'Args' encompasses all arguments required for linking device code and will +/// be parsed to generate options required to be passed into llvm-spirv tool. +static Expected<StringRef> runLLVMToSPIRVTranslation(StringRef File, + const ArgList &Args) { + llvm::TimeTraceScope TimeScope("LLVMToSPIRVTranslation"); + StringRef LLVMSPIRVPath = Args.getLastArgValue(OPT_llvm_spirv_path_EQ); + Expected<std::string> LLVMToSPIRVProg = + findProgram(Args, "llvm-spirv", {LLVMSPIRVPath}); + if (!LLVMToSPIRVProg) + return LLVMToSPIRVProg.takeError(); + + SmallVector<StringRef, 8> CmdArgs; + CmdArgs.push_back(*LLVMToSPIRVProg); + const llvm::Triple Triple(Args.getLastArgValue(OPT_triple)); + getSPIRVTransOpts(Args, CmdArgs, Triple); + StringRef LLVMToSPIRVOptions; + if (Arg *A = Args.getLastArg(OPT_llvm_spirv_options_EQ)) + LLVMToSPIRVOptions = A->getValue(); + LLVMToSPIRVOptions.split(CmdArgs, " ", /* MaxSplit = */ -1, + /* KeepEmpty = */ false); + CmdArgs.append({"-o", OutputFile}); + CmdArgs.push_back(File); + if (Error Err = executeCommands(*LLVMToSPIRVProg, CmdArgs)) + return std::move(Err); + + if (!SPIRVDumpDir.empty()) { + std::error_code EC = + llvm::sys::fs::create_directory(SPIRVDumpDir, /*IgnoreExisting*/ true); + if (EC) + return createStringError( + EC, + formatv("failed to create dump directory. path: {0}, error_code: {1}", + SPIRVDumpDir, EC.value())); + + StringRef Path = OutputFile; + StringRef Filename = llvm::sys::path::filename(Path); + SmallString<128> CopyPath = SPIRVDumpDir; + CopyPath.append(Filename); + EC = llvm::sys::fs::copy_file(Path, CopyPath); + if (EC) + return createStringError( + EC, + formatv( + "failed to copy file. original: {0}, copy: {1}, error_code: {2}", + Path, CopyPath, EC.value())); + } + + return OutputFile; +} + +Error runSYCLLink(ArrayRef<std::string> Files, const ArgList &Args) { + llvm::TimeTraceScope TimeScope("SYCLDeviceLink"); + // First llvm-link step + auto LinkedFile = linkDeviceInputFiles(Files, Args); + if (!LinkedFile) + reportError(LinkedFile.takeError()); + + // second llvm-link step + auto DeviceLinkedFile = linkDeviceLibFiles(*LinkedFile, Args); + if (!DeviceLinkedFile) + reportError(DeviceLinkedFile.takeError()); + + // LLVM to SPIR-V translation step + auto SPVFile = runLLVMToSPIRVTranslation(*DeviceLinkedFile, Args); + if (!SPVFile) + return SPVFile.takeError(); + return Error::success(); +} + +} // namespace + +int main(int argc, char **argv) { + InitLLVM X(argc, argv); + + Executable = argv[0]; + sys::PrintStackTraceOnErrorSignal(argv[0]); + + const OptTable &Tbl = getOptTable(); + BumpPtrAllocator Alloc; + StringSaver Saver(Alloc); + auto Args = Tbl.parseArgs(argc, argv, OPT_INVALID, Saver, [&](StringRef Err) { + reportError(createStringError(inconvertibleErrorCode(), Err)); + }); + + if (Args.hasArg(OPT_help) || Args.hasArg(OPT_help_hidden)) { + Tbl.printHelp( + outs(), "clang-sycl-linker [options] <options to sycl link steps>", + "A utility that wraps around several steps required to link SYCL " + "device files.\n" + "This enables LLVM IR linking, post-linking and code generation for " + "SYCL targets.", + Args.hasArg(OPT_help_hidden), Args.hasArg(OPT_help_hidden)); + return EXIT_SUCCESS; + } + + if (Args.hasArg(OPT_version)) + printVersion(outs()); + + Verbose = Args.hasArg(OPT_verbose); + DryRun = Args.hasArg(OPT_dry_run); + SaveTemps = Args.hasArg(OPT_save_temps); + + OutputFile = "a.spv"; + if (Args.hasArg(OPT_o)) + OutputFile = Args.getLastArgValue(OPT_o); + + if (Args.hasArg(OPT_spirv_dump_device_code_EQ)) { + Arg *A = Args.getLastArg(OPT_spirv_dump_device_code_EQ); + SmallString<128> Dir(A->getValue()); + if (Dir.empty()) + llvm::sys::path::native(Dir = "./"); + else + Dir.append(llvm::sys::path::get_separator()); + + SPIRVDumpDir = Dir; + } + + // Get the input files to pass to the linking stage. + auto FilesOrErr = getInput(Args); + if (!FilesOrErr) + reportError(FilesOrErr.takeError()); + + // Run SYCL linking process on the generated inputs. + if (Error Err = runSYCLLink(*FilesOrErr, Args)) + reportError(std::move(Err)); + + // Remove the temporary files created. + if (!Args.hasArg(OPT_save_temps)) + for (const auto &TempFile : TempFiles) + if (std::error_code EC = sys::fs::remove(TempFile)) + reportError(createFileError(TempFile, EC)); + + return EXIT_SUCCESS; +} diff --git a/clang/tools/clang-sycl-linker/SYCLLinkOpts.td b/clang/tools/clang-sycl-linker/SYCLLinkOpts.td new file mode 100644 index 00000000000000..959fd6c3e867cc --- /dev/null +++ b/clang/tools/clang-sycl-linker/SYCLLinkOpts.td @@ -0,0 +1,52 @@ +include "llvm/Option/OptParser.td" + +def LinkerOnlyOption : OptionFlag; + +def help : Flag<["-", "--"], "help">, + HelpText<"Display available options (--help-hidden for more)">; + +def help_hidden : Flag<["-", "--"], "help-hidden">, + HelpText<"Display all available options">; + +def verbose : Flag<["-"], "v">, HelpText<"Print verbose information">; +def version : Flag<["--"], "version">, + HelpText<"Display the version number and exit">; + +def o : JoinedOrSeparate<["-"], "o">, MetaVarName<"<path>">, + HelpText<"Path to file to write output">; +def output : Separate<["--"], "output-file">, Alias<o>, Flags<[HelpHidden]>, + HelpText<"Alias for -o">; + +def library_path_EQ : Joined<["--", "-"], "library-path=">, + Flags<[HelpHidden]>, HelpText<"Add <dir> to the library search path">; + +def device_libs_EQ : CommaJoined<["--", "-"], "device-libs=">, + Flags<[LinkerOnlyOption]>, + HelpText<"A comma separated list of device libraries that are linked during the device link.">; + +def triple : Joined<["--"], "triple">, + HelpText<"The device target triple">; +def arch : Separate<["--", "-"], "arch">, + HelpText<"Specify the name of the target architecture.">; + +def save_temps : Flag<["--", "-"], "save-temps">, + Flags<[LinkerOnlyOption]>, HelpText<"Save intermediate results">; + +def dry_run : Flag<["--", "-"], "dry-run">, Flags<[LinkerOnlyOption]>, + HelpText<"Print generated commands without running.">; + +def spirv_dump_device_code_EQ : Joined<["--", "-"], "spirv-dump-device-code=">, + Flags<[LinkerOnlyOption]>, + HelpText<"Path to the folder where the tool dumps SPIR-V device code. Other formats aren't dumped.">; + +def is_windows_msvc_env : Flag<["--", "-"], "is-windows-msvc-env">, + Flags<[LinkerOnlyOption, HelpHidden]>; + +def llvm_spirv_path_EQ : Joined<["--"], "llvm-spirv-path=">, + Flags<[LinkerOnlyOption]>, MetaVarName<"<dir>">, + HelpText<"Set the system llvm-spirv path">; + +// Options to pass to llvm-spirv tool +def llvm_spirv_options_EQ : Joined<["--", "-"], "llvm-spirv-options=">, + Flags<[LinkerOnlyOption]>, + HelpText<"Options that will control llvm-spirv step">; _______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
