On 11/3/20 2:13 PM, Nathan Sidwell wrote:
> c++ modules creates 2 new kinds of preprocessor lines
> [export] module ...
> [export] import ...
>
> To all intents and purposes these are cppdirectives spelt without a
> leading '#'. module and import are context-sensitive keywords. Thus
> preprocessor tokenizing needs a bit of token peeking. This is that
> peeking.
>
> We have a new node flag 'NODE_MODULE', which marks whether an
> identifier is significant to this peeking. When we see such an
> identifier at the beginning of a logical line, we need to peek further
> and figure out whether these are those keywords.
>
> When successfully peeked, we replace the identifiers with
> internal-only tokens that the c++ parser recognizes.
>
>
> 04-cpp-lexer.diff
>
> diff --git c/libcpp/include/cpplib.h w/libcpp/include/cpplib.h
> index 8e398863cf6..81be6457951 100644
> --- c/libcpp/include/cpplib.h
> +++ w/libcpp/include/cpplib.h
>
> @@ -888,9 +915,9 @@ struct GTY(()) cpp_hashnode {
> unsigned int directive_index : 7; /* If is_directive,
> then index into directive table.
> Otherwise, a NODE_OPERATOR. */
> - unsigned char rid_code; /* Rid code - for front ends. */
> + unsigned int rid_code : 8; /* Rid code - for front ends. */
> + unsigned int flags : 9; /* CPP flags. */
> ENUM_BITFIELD(node_type) type : 2; /* CPP node type. */
> - unsigned int flags : 8; /* CPP flags. */
>
> /* 6 bits spare (plus another 32 on 64-bit hosts). */
Someone already mentioned it, but the # of spare bits needs updating.
>
> diff --git c/libcpp/lex.c w/libcpp/lex.c
> index fb222924c8c..b3498f195bf 100644
> --- c/libcpp/lex.c
> +++ w/libcpp/lex.c
> @@ -2606,6 +2622,131 @@ _cpp_temp_token (cpp_reader *pfile)
> return result;
> }
>
> +/* RESULT is a CPP_NAME with NODE_MODULE set. See if we should enter
> + deferred_pragma mode to tokenize the rest of the line. */
> +
> +static void
> +cpp_maybe_module_directive (cpp_reader *pfile, cpp_token *result)
> +{
> + unsigned backup = 0; /* Tokens we peeked. */
> + cpp_hashnode *node = result->val.node.node;
> + cpp_token *peek = result;
> + cpp_token *keyword = peek;
> + cpp_hashnode *(&n_modules)[spec_nodes::M_HWM][2] =
> pfile->spec_nodes.n_modules;
> + int header_count = 0;
> +
> + /* Enter directives mode for the peeking. */
> + pfile->state.in_deferred_pragma = true;
> + pfile->state.pragma_allow_expansion = true;
> + pfile->state.save_comments = 0;
> + pfile->directive_line = result->src_loc;
It looks like you slam in known values when you drop out of directive
mode rather than restoring the original values. That may be OK, I'm not
all that familiar with this code to know if that's corerct or not.
> +
> + if (__builtin_expect (node == n_modules[spec_nodes::M__IMPORT][0], false))
> + /* __import */
> + header_count = backup + 2 + 16;
> + else if (__builtin_expect (node == n_modules[spec_nodes::M_IMPORT][0],
> false))
> + /* import */
> + header_count = backup + 2 + (CPP_OPTION (pfile, preprocessed) ? 16 : 0);
> + else if (__builtin_expect (node == n_modules[spec_nodes::M_MODULE][0],
> false))
> + ; /* module */
> + else
> + goto not_module;
Are the __builtin_expects really useful here? If not I'd prefer to
avoid them and just let the predictors do their job. Similarly
elsewhere in this patch.
+ {
> + not_module:
> + /* Drop out of directive mode. */
> + pfile->state.save_comments
> + = !CPP_OPTION (pfile, discard_comments);
> + pfile->state.in_deferred_pragma = false;
> + pfile->state.angled_headers = false;
This doesn't seem to match the code to go into directives mode all that
well. You don't reset pragma_allow_expansion or directive_line.
Jeff
