On Thu, Jul 24, 2025 at 6:11 PM Martin Uecker <ma.uec...@gmail.com> wrote:
>
> Am Donnerstag, dem 24.07.2025 um 16:26 +0000 schrieb Aaron Ballman:
> > On Thu, Jul 24, 2025 at 3:03 PM Martin Uecker <ma.uec...@gmail.com> wrote:
> > >
> > > Am Donnerstag, dem 24.07.2025 um 14:08 +0000 schrieb Aaron Ballman:
> > > > On Wed, Jul 23, 2025 at 8:38 PM Martin Uecker <ma.uec...@gmail.com>
> > > > wrote:
> > > >
> > >
> ...
> > > >
> > > > Personally, I'm not excited by it because one of the big sticking
> > > > points in the Clang community is shared header files with C++. Because
> > > > these attributes are used on structures and functions, the two most
> > > > common things you'll find in a shared header file, we *really* want
> > > > the feature to be workable in both languages to the greatest extent
> > > > possible. And once we care about C++, things get so much harder due to
> > > > the extra complexity it brings. So, for example, we'd have to figure
> > > > out how to handle things like:
> > > > ```
> > > > template <typename Ty>
> > > > struct S {
> > > > char *buffer __counted_by_expr(Ty len; len + 7);
> > > > int len; // Oooooops
> > > > };
> > > >
> > > > template <typename Ty, typename Uy>
> > > > struct T {
> > > > char *buffer __counted_by_expr(Ty len; len + 7);
> > > > Uy len; // Grrrrr
> > > > };
> > > > ```
> > >
> > > For my understanding: What is the problem here? I would be an
> > > error if the declared type of len is inconsistent between the
> > > attribute and the type that cames later in the member. I guess
> > > a compiler could also warn already when it sees a template
> > > like this where it refers to different template arguments.
> > >
> > > But then, templates also certainly do not appear in shared headers,
> > > so I am not sure why Clang could not simmply offer both,
> > > a late-parsing version and also a C-compatible __counted_by_expr.
> > >
> > > I can understand if this is not your first choice, but it seems
> > > to be a reasonable compromise to me.
> >
> > Ah, apologies, I wasn't clear. My thinking is: we're (Clang folks)
> > going to want it to work in C++ mode because of shared headers. If it
> > works in C++ mode, then we have to figure out what it means with all
> > the various C++ features that are possible, not just the use cases
> > that appear within a shared header. That's how I got onto templates as
> > just one example of where we'd have to figure out what the behavior
> > should be. Other questions come up outside of templates as well (use
> > in default arguments, lambda captures, use of types with conversion
> > operators, etc).
>
> Ok, but this assumes that clang does *not* offer two versions, but
> only one that then needs to work perfectly in all possible contexts.
> This certainly makes it much more challenging to find a solution.
Oh, sorry, I think I missed that part of the design discussion!
> If clang would support __counted_by with all the semantics you like
> for C++ and also __counted_by_expr for GCC / C compatibility, then
> you could simply forbid __counted_by_expr in templates or other
> C++ scenarious - if it happens to cause problems.
I'll have to think about that more, but my initial reaction is: that's
making our implementation/design problems be the user's problem. Maybe
that's fine? But it would be kind of frustrating as a user to have
code using `__counted_by(foo)` that I want to modify to say
`__counted_by(foo * CHAR_BIT)` but then find out it needs to be
`__counted_by_expr(foo * CHAR_BIT)` and now there are contexts where
that doesn't work because of potential C++ shenanigans.
> But then, I also I do not really understand why you think
> __counted_by_expr has more complicated interactions with C++ than
> delayed-parsing counted_by (see below).
>
>
> > I think some of these questions may even apply in C.
> > Like... is this valid?
> >
> > void func(char *buffer [[counted_by(int N[12]; N[10])]], int N[8]);
> >
> > Or this?
> >
> > void func(char *buffer [[counted_by(int N[*]; N[10])]], int N[*]); /*
> > Can't usually spelling int N[*] outside of a parameter list, but this
> > is a parameter list of sorts, so maybe it's fine? */
>
> Any language feature would need a specification. In the discussion
> with Bill it was became clear that we would need to add
> constraints to a delayed-parsing version, which somebody would
> need to spell out and implement. so I do not see how this is an
> argument for or against a specific solution.
It's pointing out the kinds of design concerns I see with the
approach. How the design concerns are resolved factors into whether
I'm for or against.
> > I'm not saying it's not a reasonable compromise, FWIW. More just... I
> > think allowing for two declarations increases the complexity of the
> > feature in ways that aren't well understood yet.
>
> This is one thing where we are not on the same page. For me,
> delayed-parsing is the feature that is far more complex also
> in terms of interaction with the rest of the language.
>
> Forward declarations may look more complex, but it unambigously
> identifies the object and constrains the type, which makes things
> simpler IMHO.
Yeah, I think we're not on the same page here as well. If used in the
common case, I can see how it unambiguously identifies the object and
constrains the type. It's the uncommon cases that I am worried about
(because if it's syntactically possible to spell, my experience is
that users will do it and file bug reports about it "not working").
What I mean by common case are straightforward uses like: void
func(int n; char *buffer __counted_by(n), int n); with no name hiding
via other declarations, which are the majority of uses. I think any
choice of forward declarations, .N, late parsing, etc work for the
simple cases.
For the uncommon cases, I think late parsing ends up with less
complexity for the user to deal with; most of the problems boil down
to ambiguous names (or am I wrong there?). Both C and C++ users are
already used to dealing with naming conflicts (in different ways) and
so I think it's less burden on them to say "and here's one more place
you can have name hiding" and handle ambiguities with diagnostics. And
the added benefit is that for the straightforward case, when there's
no name hiding (which Apple's usage experience suggests is a rarity in
practice), late parsing requires the user to write less code, which I
think is valuable:
void func(int n; char *buffer __counted_by(n), int n);
vs
void func(char *buffer __counted_by(n), int n);.
(FWIW, I consider .N to be late parsing as well because the only way I
think it can work in practice is if the type information is
available.)
> > > > I think it's possible to handle these situations, but we'd have to sit
> > > > down and think through all the edge cases and whether we can handle
> > > > them all with some reasonable QoI. I think we'd ultimately run into
> > > > the same concerns here as we ran into with forward declared
> > > > parameters. I think the reason folks in Clang are more comfortable
> > > > with late parsing is because it means the user doesn't have to repeat
> > > > the type and the name which makes for less chances for the user to
> > > > screw things up and get into these weird situations. There can be
> > > > other weird situations with late parsing too, of course, but I think
> > > > the scope of those edge cases is a bit narrower.
> > >
> > > TBH, I am not terrible convinced about this argument.
> > >
> > > If I understood it correctly, the late parsing design seems to make
> > > no distinctions between which identifiers is used, the local or
> > > the global one and just prefers the local one if it exists, possibly
> > > giving a warning if there is also a global one.
> >
> > I think I'd describe it as following typical lexical scoping behavior
> > -- the closest declaration of the identifier is what's found by the
> > lookup.
>
> I am not sure what you mean by "typical lexical scoping behavior".
>
> The scoping behavior that C - so far - consistently uses everywhere
> works not like this (except for labels, which have no type).
int n; // #4
void func(int n /* #3 */, int array[n /* Matches with #3 if it's
declared, #4 otherwise */]) {
int n; // #2
{
int n; // #1
n = 12; // Matches with #1
}
n = 100; // Matches with #2 if it's declared, #3 otherwise
}
struct S {
int n; // #5
int array[n]; // Matches #5 in C++, #4 in C, this is the interesting case.
};
(#2 and #3 create a redefinition, so assume those are mutually
exclusive declarations) In all cases except the interesting one, the
use of the identifier matches with the declaration from the nearest
scope. I would argue that making the interesting case behave the same
in C as it does in C++ would make C more consistent than it is today
because it would pick the closest declaration, same as what happens
within a function body or compound scope.
> It also does not seem typical for C++, e.g. the
> following gives an error:
>
> struct foo {
> char buf[E];
> static const int E = 1;
> };
>
> https://godbolt.org/z/WTvcf7hza
C++ has this notion of "complete-class context"
(https://eel.is/c++draft/class.mem.general#10) which identifies where
late parsing is used. Within an array extent, it's not a complete
class context. Confusingly, within a default initializer, it *is*:
https://godbolt.org/z/s71aW1z33
(I think if we went down the route of late parsing for bounds safety
attributes, WG21 should make any use within a declaration of a data
member be within a complete class context. The current situation
seems... not ideal IMO.)
> > But in the event that causes a different lookup result from
> > what the current standard behavior would give you, it should be
> > diagnosed. Personally, I'd feel most comfortable if that diagnostic
> > was a warning which defaults to an error; basically, make the user
> > decide how to handle it on a case by case basis but "standard behavior
> > wins" if you disable the diagnostic.
>
> The reason I am not a fan of introducing a potential ambiguity and then
> have warnings/error to avoid problems is that this makes the code fragile.
> Introducing a name in some enclosing context (e.g. by including a header)
> will break the local code that uses the same name, even though it is
> unrelated. Scoping is meant to prevent exactly this!
Yeah, that is a problem. But it's not a new problem, right? The same
fragility happens with macros. Though, not making that fragility worse
would be a good thing, it's not like users appreciate unintentional
conflicts with macro names.
> I also like to point out that clang is not currently following this
> philosophy. In fact, it is that GCC rejects the following code in C++,
> while clang compiles it - but also not using the scoping rules
> proposed here, but the same one C uses!
It's not that we're not following the philosophy, it's that bounds
safety work is still in-progress and we're waiting for a resolution to
this problem before moving forward with changes upstream. If late
parsing ends up being the approach that everyone can live with, I'd
expect this behavior to be updated in Clang accordingly
~Aaron
>
> constexpr int E = 1;
>
> struct foo {
> char buf[E];
> static const int E = 3;
> static_assert(1 == sizeof(buf));
> };
>
> https://godbolt.org/z/zqWafsGh5.
>
> If you remove the file-scope E, both compilers reject this code.
> Neither clang nor GCC refer to the later identifier.
>
>
> >
> > > My C++ examples shows that you can easily run into UB here in C++,
> > > especially since subtle differnt rules apply in different but very
> > > similar scenarious. How can this not be error prone?
> > >
> > > The forward declaration, the [.N] syntax, and also __self__ etc.
> > > would all make this explicit which identifiers is meant.
> >
> > I think they come with tradeoffs but so far, everything seems to be
> > error prone in different ways. :-(
> >
> > > > The other downside is that we have more attributes that need to
> > > > support something similar, like the thread safety attributes (which I
> > > > believe is also an important use case for the Linux kernel folks?). We
> > > > could do this dance on a per-attribute basis, but if the solution
> > > > worked for all attributes *and* array extents at the same time, that
> > > > would be nice. Not certain it's a requirement though.
> > >
> > > True. But if it is to work properly for arrays in C too, then the
> > > C constraints are also important IMHO, not just the C++ rules.
> >
> > Agreed! I don't think "C++ rules should win, that's the end of the
> > discussion" is tenable; I think it's more that we need to handle all
> > the oddities of both languages otherwise we're going to end up with
> > something like C99's array parameter features that didn't get adopted
> > into C++ (e.g., [static 12] or [*], etc). I think that hampered their
> > adoption in the wild at least in part because of the cross-language
> > issues.
>
>
> In fact, one of best thing Clang / GCC could do for language
> interoperability would be to support [*] and [n] in C++ in
> function declarations, even if it is just a no op.
>
> >
> > > > > The the thing is that WG14 had (weak) consensus for parameter
> > > > > forward declarations and I think more consensus for [.N]
> > > > > syntax in structures already. So I had hoped that we will be
> > > > > able to make progress on this.
> > > >
> > > > Question on the .N syntax: I thought I heard that this was something
> > > > GCC could handle, but that it still requires late parsing to ensure
> > > > type information for N is available and that was a problem. e.g.,
> > > >
> > > > void func(char *buffer __counted_by(.N * sizeof(.N)), int N);
> > >
> > > >
> > > > where we'd need to know both the name and the type. Am I wrong about
> > > > that being a challenge for GCC to support?
> > >
> > > I think it is generally a challenge to support.
> >
> > Thanks for the confirmation!
> >
> > > One could certainly
> > > store away the tokens and parse them later (this is certainly doable),
> > > but it adds a lot of issues because you need to add a lot of constraints
> > > for things which should then not be allwoed. And it is still not an
> > > acceptable solution for size arguments in C.
> >
> > Yeah, that's basically the same gripe I have about having forward
> > declarations; we have to figure out all the weird edge cases and what
> > constraints are necessary to have decent QoI.
> >
> > > .N would work here if you combine with a rule such as ".N" is always
> > > converted to "size_t". Or you require an explicit cast if is different
> > > to "size_t" .
> >
> > I think converting .N to size_t could work if the feature was limited
> > to just bounds safety, but it would be unusable for things like thread
> > safety attributes where the argument needs to be some kind of
> > mutex-like object. But even for bounds safety, I think we end up with
> > really weird behavior: void func(char *buffer __counted_by(sizeof(.N)
> > /* sizeof(size_t) */), typeof(.N) x /* size_t x */, int N /* wait,
> > what? */);
>
> I agree, this is why I would favour GCC's forward declaration for
> parameters. In structures I think .N might be ok, because you want
> to restrict the expressions anyhow, as they are evaluated implicitely
> on each member access.
>
> >
> > If we were to require explicit casts... I think we're back to the same
> > thing as having forward declarations in terms of concerns because
> > that's what opens up the possibility of type mismatches.
>
> I am not sure I understand the concern. You can have type errors
> in all expressions.
>
> The key insight is that the cast would hide the original type, so
> you do not need to know it when parsing the expression
>
> struct foo {
> int *buf __countey_by((size_t)N);
> int N;
> };
>
> You only need to later check that type you actually have can be
> converted to the type used in the cast, but you need to support
> this anyhow!
>
> >
> > I guess the way I see it is:
> >
> > If there's only one declaration involved (late parsing approach), then
> > there's potential name lookup confusion which I think is worse in C
> > than it is in C++.
>
> I would value consistency in C more, but I agree that for the of
> one identifer late parsing is not causing confusion or implementation
> issues. But only if you look at this by itself, once you mix it
> with other language features that is already not clear anymore:
>
> constexpr size_t size = 4;
> struct foo {
> char (*buf)[size] __counted_by(size); // two different "size"!
> int size;
> };
>
> > If there's more than one declaration involved (any kind of forward
> > declaration syntax, use of .N with explicit casts, etc), then there's
> > potential type confusion which I think is worse in C++ than it is in
> > C.
> >
> > Either one is going to need constraints to help with the confusion. I
> > don't know which one ends up with less constraints or more readable
> > code.
>
> Martin
>
>
> > >
>
