On Mon, Aug 16, 2021 at 06:07:57PM -0400, Jason Merrill wrote:
> > It is unclear if it would be enough
> > to remove just one or if all padding tokens should be removed.
> > Anyway, e.g. the previous removal of all padding tokens at the end of
> > __VA_OPT__ is undesirable, as it e.g. eats also the padding tokens needed
> > for the H4 example from the paper.
>
> Hmm, I don't see why. Looking at the H4 example, it seems that the
> expansion of __VA_OPT__ should be
>
> a <placemarker>
>
> so when we paste to b, b is pasted to the placemarker, leaving a as a
> separate token.
#define H4(X, ...) __VA_OPT__(a X ## X) ## b
H4(, 1) // replaced by a b
We actually get with vanilla trunk
a <placemarker> <placemarker>
where the former comes from:
2216 /* Padding on the left of an argument (unless RHS of ##). */
2217 if ((!pfile->state.in_directive ||
pfile->state.directive_wants_padding)
2218 && src != macro->exp.tokens && !(src[-1].flags & PASTE_LEFT)
2219 && !last_token_is (buff, vaopt_start))
2220 {
2221 const cpp_token *t = padding_token (pfile, src);
2222 unsigned index = expanded_token_index (pfile, macro, src, i);
2223 /* Allocate a virtual location for the padding token and
2224 append the token and its location to BUFF and
2225 VIRT_LOCS. */
2226 tokens_buff_add_token (buff, virt_locs, t,
2227 t->src_loc, t->src_loc,
2228 map, index);
2229 }
and the latter one is added at
2303 /* Avoid paste on RHS (even case count == 0). */
2304 if (!pfile->state.in_directive && !(src->flags & PASTE_LEFT)
2305 && !last_token_is (buff, vaopt_start))
2306 {
2307 const cpp_token *t = &pfile->avoid_paste;
2308 tokens_buff_add_token (buff, virt_locs,
2309 t, t->src_loc, t->src_loc,
2310 NULL, 0);
2311 }
and trunk eats both <placemarker>s in:
/* Remove any tail padding from inside the __VA_OPT__. */
paste_flag = tokens_buff_last_token_ptr (buff);
while (paste_flag && paste_flag != start
&& (*paste_flag)->type == CPP_PADDING)
{
tokens_buff_remove_last_token (buff);
paste_flag = tokens_buff_last_token_ptr (buff);
}
and thus H4(, 1) is replaced by ab instead of the right a b.
We want to remove the latter <placemarker> but not the former one, and
the patch adds the vaopt_padding_tokens counter for it to control
how many placemarkers are removed on vaopt_state::END.
As can be seen in #c1 and #c2 of the PR, I've tried various approaches,
but neither worked out for all the cases except the posted one.
Jakub
