jansvoboda11 updated this revision to Diff 321103.
jansvoboda11 added a comment.
Herald added a reviewer: jdoerfert.
Herald added a subscriber: sstefan1.
Polish, document ImpliedByAnyOf and ShouldParse
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D95790/new/
https://reviews.llvm.org/D95790
Files:
clang/docs/InternalsManual.rst
Index: clang/docs/InternalsManual.rst
===================================================================
--- clang/docs/InternalsManual.rst
+++ clang/docs/InternalsManual.rst
@@ -572,6 +572,275 @@
The Frontend library contains functionality useful for building tools on top of
the Clang libraries, for example several methods for outputting diagnostics.
+Compiler Invocation
+-------------------
+
+One of the classes provided by the Frontend library is ``CompilerInvocation``,
+which holds information that describe current invocation of the Clang frontend.
+The information typically comes from the command line constructed by the Clang
+driver, or directly from clients performing custom initialization. The data
+structure is split into logical units used by different parts of the compiler,
+for example ``PreprocessorOptions``, ``LanguageOptions`` or ``CodeGenOptions``.
+
+Command Line Interface
+----------------------
+
+The command line interface of the Clang ``-cc1`` frontend is defined alongside
+the driver options in ``clang/Driver/Options.td``. The information making up an
+option definition include the name and prefix (for example ``-std=``), form and
+position of the option value, help text, aliases and more. Each option may
+belong to a certain group and can be marked with zero or more flags. Options
+accepted by the ``-cc1`` frontend are marked with the ``CC1Option`` flag.
+
+Command Line Parsing
+--------------------
+
+Option definitions are processed by the ``-gen-opt-parser-defs`` tablegen
+backend, transformed into a list of invocations of the ``OPTION`` macro and
+stored in ``clang/Driver/Driver.inc``. This file is then used to create instance
+of ``llvm::opt::OptTable``, which acts as a command line preprocessor. The
+preprocessed command line is stored in ``llvm::opt::ArgList``, an object that
+provides an API for performing simple queries on the contents of the command
+line.
+
+Finally, the ``CompilerInvocation::CreateFromArgs`` function is responsible for
+the actual parsing of command line arguments. It maps the contents of the
+``ArgList`` onto fields of ``CompilerInvocation``, normalizing the values in the
+process.
+
+Command Line Generation
+-----------------------
+
+Any valid ``CompilerInvocation`` created from a ``-cc1`` command line can be
+also serialized back into semantically equivalent command line in a
+deterministic manner. This enables features such as implicitly discovered,
+explicitly built modules.
+
+..
+ TODO: Create and link corresponding section in Modules.rst.
+
+Option Marshalling infrastructure
+---------------------------------
+
+The obvious way to parse command line arguments is to write the code that
+queries ``ArgList`` and constructs ``CompilerInvocation`` manually. However,
+given that behavior of most command line options is the same, this approach
+would result in lots of repetitive code. This is the reason why most of the
+code for parsing and generating arguments is automatically generated from
+``Marshalling`` annotations on the tablegen option definitions. This section
+goes through the basics of the automatic marshalling system.
+
+To read and modify contents of ``CompilerInvocation``, the system uses key
+paths, which are declared in two steps. First, a tablegen definition for the
+``CompilerInvocation`` member is created by inheriting from ``KeyPathAndMacro``.
+
+.. code-block::
+
+ // Options.td
+
+ class LangOpts<string field> : KeyPathAndMacro<"LangOpts->", field, "LANG_"> {}
+ // CompilerInvocation member ^^^^^^^^^^
+ // OPTION_WITH_MARSHALLING prefix ^^^^^
+
+The first argument to the parent class is the beginning of the key path that
+references the ``CompilerInvocation`` member. This argument ends with ``->`` if
+the member is a pointer type or with ``.`` if it's a value type. The second
+argument is the only parameter passed to the child class. The child class can be
+used like so: ``LangOpts<"IgnoreExceptions">``, constructing a key path to the
+field ``LangOpts->IgnoreExceptions``. The third argument passed to the parent
+class is a string that the tablegen backend uses as a prefix to the
+``OPTION_WITH_MARSHALLING``. Using the key path then instructs the backend to
+generate the following code.
+
+.. code-block:: c++
+
+ // Options.inc
+
+ #ifdef LANG_OPTION_WITH_MARSHALLING
+ LANG_OPTION_WITH_MARSHALLING([...], LangOpts->IgnoreExceptions, [...])
+ #endif // LANG_OPTION_WITH_MARSHALLING
+
+Such definition can be used used in the parsing and generating functions:
+
+.. code-block:: c++
+
+ // CompilerInvoation.cpp
+
+ bool CompilerInvocation::ParseLangArgs(LangOptions *LangOpts, ArgList &Args,
+ DiagnosticsEngine &Diags) {
+ bool Success = true;
+
+ #define LANG_OPTION_WITH_MARSHALLING( \
+ PREFIX_TYPE, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM, \
+ HELPTEXT, METAVAR, VALUES, SPELLING, SHOULD_PARSE, ALWAYS_EMIT, KEYPATH, \
+ DEFAULT_VALUE, IMPLIED_CHECK, IMPLIED_VALUE, NORMALIZER, DENORMALIZER, \
+ MERGER, EXTRACTOR, TABLE_INDEX) \
+ PARSE_OPTION_WITH_MARSHALLING(Args, Diags, Success, ID, FLAGS, PARAM, \
+ SHOULD_PARSE, KEYPATH, DEFAULT_VALUE, \
+ IMPLIED_CHECK, IMPLIED_VALUE, NORMALIZER, \
+ MERGER, TABLE_INDEX)
+ #include "clang/Driver/Options.inc"
+ #undef LANG_OPTION_WITH_MARSHALLING
+
+ // ...
+
+ return Success;
+ }
+
+ void CompilerInvocation::GenerateLangArgs(LangOptions *LangOpts,
+ SmallVectorImpl<const char *> &Args,
+ StringAllocator SA) {
+ #define LANG_OPTION_WITH_MARSHALLING( \
+ PREFIX_TYPE, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM, \
+ HELPTEXT, METAVAR, VALUES, SPELLING, SHOULD_PARSE, ALWAYS_EMIT, KEYPATH, \
+ DEFAULT_VALUE, IMPLIED_CHECK, IMPLIED_VALUE, NORMALIZER, DENORMALIZER, \
+ MERGER, EXTRACTOR, TABLE_INDEX) \
+ GENERATE_OPTION_WITH_MARSHALLING( \
+ Args, SA, KIND, FLAGS, SPELLING, ALWAYS_EMIT, KEYPATH, DEFAULT_VALUE, \
+ IMPLIED_CHECK, IMPLIED_VALUE, DENORMALIZER, EXTRACTOR, TABLE_INDEX)
+ #include "clang/Driver/Options.inc"
+ #undef LANG_OPTION_WITH_MARSHALLING
+
+ // ...
+ }
+
+How does the tablegen backend know what to put in place of ``[...]`` in the
+generated file? This is specified by the ``Marshalling`` utilities described
+below. All of them take a key path argument and possibly other information
+required for parsing or generating the command line argument.
+
+**Positive Flag**
+
+The key path defaults to ``false`` and is set to ``true`` when the flag is
+present on command line.
+
+.. code-block::
+
+ def fignore_exceptions : Flag<["-"], "fignore-exceptions">, Flags<[CC1Option]>,
+ MarshallingInfoFlag<LangOpts<"IgnoreExceptions">>;
+
+**Negative Flag**
+
+The key path defaults to ``true`` and is set to ``false`` when the flag is
+present on command line.
+
+.. code-block::
+
+ def fno_verbose_asm : Flag<["-"], "fno-verbose-asm">, Flags<[CC1Option]>,
+ MarshallingInfoNegativeFlag<CodeGenOpts<"AsmVerbose">>;
+
+**Negative and Positive Flag**
+
+The key path defaults to the specified value (``false``, ``true`` or some value
+that's statically unknown in the tablegen file). Then, the key path is set to
+the value associated with the flag that appears last on command line.
+
+.. code-block::
+
+ defm legacy_pass_manager : BoolOption<"f", "legacy-pass-manager",
+ CodeGenOpts<"LegacyPassManager">, DefaultFalse,
+ PosFlag<SetTrue, [], "Use the legacy pass manager in LLVM">,
+ NegFlag<SetFalse, [], "Use the new pass manager in LLVM">,
+ BothFlags<[CC1Option]>>;
+
+With most such pair of flags, the ``-cc1`` frontend accepts only the flag that
+changes the default key path value. The Clang driver is responsible for
+accepting both and either forwarding the changing flag or discarding the flag
+that would just set the key path to its default.
+
+The first argument to ``BoolOption`` is a prefix that is used to construct the
+full names of both flags. The positive flag would then be named
+``flegacy-pass-manager`` and the negative ``fno-legacy-pass-manager``.
+``BoolOption`` also implies the ``-`` prefix for both flags. It's also possible
+to use ``BoolFOption`` that implies the ``"f"`` prefix and ``Group<f_Group>``.
+
+**String**
+
+The key path defaults to the specified string, or an empty one, if omitted. When
+the option appears on the command line, the argument value is simply copied.
+
+.. code-block::
+
+ def isysroot : JoinedOrSeparate<["-"], "isysroot">, Flags<[CC1Option]>,
+ MarshallingInfoString<HeaderSearchOpts<"Sysroot">, [{"/"}]>;
+
+**List of Strings**
+
+The key path defaults to an empty ``std::vector<std::string>``. Values specified
+with each appearance of the option on command line are appended to the vector.
+
+.. code-block::
+
+ def frewrite_map_file : Separate<["-"], "frewrite-map-file">, Flags<[CC1Option]>,
+ MarshallingInfoStringVector<CodeGenOpts<"RewriteMapFiles">>;
+
+**Integer**
+
+The key path defaults to the specified integer value, or ``0`` if omitted. When
+the option appears on the command line, its value gets parsed by ``llvm::APInt``
+and the result is assigned to the key path on success.
+
+.. code-block::
+
+ def mstack_probe_size : Joined<["-"], "mstack-probe-size=">, Flags<[CC1Option]>,
+ MarshallingInfoStringInt<CodeGenOpts<"StackProbeSize">, "4096">;
+
+**Enumeration**
+
+The key path defaults to the value specified in ``MarshallingInfoEnum`` prefixed
+by the contents of ``NormalizedValuesScope`` and ``::``. This ensures correct
+reference to an enum case is formed even if the enum resides in different
+namespace or is an enum class. If the value present on command line does not
+match any of the comma-separated values from ``Values``, an error diagnostics is
+issued. Otherwise, the corresponding element from ``NormalizedValues`` at the
+same index is assigned to the key path (also correctly scoped). The number of
+comma-separated string values and elements of the array within
+``NormalizedValues`` must match.
+
+.. code-block::
+
+ def mthread_model : Separate<["-"], "mthread-model">, Flags<[CC1Option]>,
+ NormalizedValuesScope<"LangOptions::ThreadModelKind">,
+ MarshallingInfoEnum<LangOpts<"ThreadModel">, "POSIX">,
+ Values<"posix,single">,
+ NormalizedValues<["POSIX", "Single"]>;
+
+..
+ Intentionally omitting MarshallingInfoBitfieldFlag. It's adding some
+ complexity to the marshalling infrastructure and might be removed.
+
+It's also possible to define relationships between options.
+
+**Implication**
+
+The key path defaults to the default value from the primary ``Marshalling``
+annotation. Then, if any of the elements of ``ImpliedByAnyOf`` evaluate to true,
+the key path value is changed to the specified value or ``true`` if missing.
+Finally, the command line is parsed according to the primary annotation.
+
+.. code-block::
+
+ def fms_extensions : Flag<["-"], "fms-extensions">, Flags<[CC1Option]>,
+ MarshallingInfoFlag<LangOpts<"MicrosoftExt">>,
+ ImpliedByAnyOf<[fms_compatibility.KeyPath], "true">;
+
+**Condition**
+
+The option is parsed only if the expression in ``ShouldParseIf`` evaluates to
+true.
+
+.. code-block::
+
+ def fopenmp_enable_irbuilder : Flag<["-"], "fopenmp-enable-irbuilder">, Flags<[CC1Option]>,
+ MarshallingInfoFlag<LangOpts<"OpenMPIRBuilder">>,
+ ShouldParseIf<fopenmp.KeyPath>;
+
+The building blocks provided by the marshalling infrastructure cover most of the
+options accepted by the ``-cc1`` frontend. It's recommended to design new
+command line options in such a way that the infrastructure can be reused and
+manual implementation of parsing and generation can be avoided. However, manual
+implementation is still possible.
+
The Lexer and Preprocessor Library
==================================
_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
