Hi John,
(I changed gcc-patches to gcc) Am Samstag, dem 15.03.2025 um 00:22 -0400 schrieb John McCall: > > On 14 Mar 2025, at 15:18, Martin Uecker wrote: > > > > ... > > > > But let's rephrase my point a bit more precisely: One could take > > > > a strict subset of C that includes variably modified types but > > > > obviously has to forbid a lot other things (e.g. arbitrary pointer > > > > conversions or unsafe down-casts and much more) and make this a > > > > memory-safe language with dependent types. This would also > > > > require adding run-time checks at certain places where there > > > > is now UB, in particular where two VLA types need to be compatible. > > Mmm. You can certainly subset C to the point that it’s memory-safe, but > > it wouldn’t really be anything like C anymore. It would not look like most legacy C code. But a subset of C would still be C and it would be compilable by existing compilers (but without safety) and can seaminglessly integrated into existing projects. > > As long as C has a heap, > > I don’t see any path to achieving temporal safety without significant > > extensions to the language. Yes, for temporal memory you would need some extensions. But the first step for me would be to make this C subset very restrictive and then incrementally extend it later. There are applications where a restricted subset would help, e.g. because dynamic memory allocation is not allowed anyway or because one is interested to only check some critical part of a program and all memory management is outside of this part. > > But if we’re just talking about spatial safety, > > then sure, that could be a lot closer to C today. Yes. > > Is that your vision, then, that you’d like to see the same sort of checks > > that -fbounds-safety does, but you want them based firmly in the language > > as a dynamic check triggered by pointer type conversion, with bounds > > specified using variably-modified types? It’s a pretty elegant vision, and > > I can see the attraction. The VLA accesses are bounds-checked via UBsan: https://godbolt.org/z/9q1f8Wx51 And for the pointer assignment checks I have a complete patch for GCC that adds this. > > It has some real merits, which I’ll get to below. > > I do see at least two significant challenges, though. > > The first and biggest problem is that, in general, array bounds can only be > > expressed on a pointer value if it’s got pointer to array type. Most C array > > code today works primarily with pointers to elements; programmers just use > > array types to create concrete arrays, and they very rarely use pointers to > > array type at all. There are a bunch of reasons for that: > > * Pointers to arrays have to be dereferenced twice: (*ptr)[idx] instead > > of ptr[idx]. > > * That makes them more error-prone, because it is easy to do pointer > > arithmetic at the wrong level, e.g. by writing ptr[idx], which will > > stride by multiples of the entire array size. That may even pass the > > compiler without complaint because of C’s laxness about conversions. This is not a problem in my experience because type checking catches this - not the indexing, but that the result type is wrong for what you do afterwards. > > * Keeping the bound around in the pointer type is more work and doesn’t do > > anything useful right now. It already gives you bounds checking with UBSan! It also makes the bounds and their relations visible to human readers or other tools. Another advantage is that one can read the bound back from the pointer type via sizeof (and in the future _Countof). I am coming more from the numerical side, where having the bounds in multi-dimensional arrays is helpful. > > * A lot of C programmers dislike nested declarator syntax and can’t > > remember > > how it works. Those of us who can write it off the top of our heads are > > quite atypical. I think they would get used to it when seeing it used elsewhere. But I agree many might not want to use it and it would also not help legacy code. So I would also very much like to see improvements in bounds checking that it is implicit or based on other annotations as in -fbounds-safety and BDOS-based checking. > > Now, there is an exception: you can write a parameter using an array type, > > and it actually declares a pointer parameter. You could imagine using this > > as a syntax for an enforceable array bound for arguments, although the > > committee did already decide that these bounds were meaningless without > > static. The plan is to strengthen the meaning of it. If the stricter mode is opt-in, this is not a major problem. > > Unfortunately, you can’t do this in any other position and still > > end up with just a pointer, so it’s not helpful as a general syntax for > > associating bounds with pointers. In existing code. I argee that there we should use attribute-based annotations. > > The upshot is that this isn’t really something people can just adopt by > > adding annotations. It’s not just a significant rewrite, it’s a rewrite that > > programmers will have very legitimate objections to. I think that makes this > > at best a complement to the “sidecar” approach taken by -fbounds-safety > > where we can track top-level bounds to a specific pointer value. I agree we should do both. I am more optimistic than you about programmers using it though. After all, ti me the code looks more elegant than adding attributes and there is also some aversion of having things done behind your back. > > The second problem is that there are some extralingual problems that > > -fbounds-safety has to solve around bounds that aren’t just local > > evaluations of bounds expressions, and a type-conversion-driven approach > > doesn’t help with any of them. > > As you mentioned, the design of variably-modified types is based on > > evaluating the bounds expression at some specific point in the program > > execution. Since these types can only be written locally, the evaluation > > point is obvious. If we wanted to dynamically enforce bounds during > > initialization, it would simply be another use of the same computed bound: > > int count = ...; int (*ptr)[count * 10] = source_ptr; > > Here we would evaluate count * 10 exactly once and use it both as (1) part > > of the destination type when initializing ptr with source_ptr and (2) > > part of the type of ptr for all uses of it. For example, if source_ptr > > were of type int (*)[100], we would dynamically check that > > count * 10 <= 100. This all works perfectly with an arbitrary bounds > > expression; it could even contain an opaque function call. My patch to GCC checks that the bound in source is the same as in ptr, e.g. int count1 = 10; int count2 = 9; int (*source_ptr)[count1] = malloc(sizeof *source_ptr); int (*ptr)[count2] = source_ptr; would trap. > > Note that we don’t need any special behavior specifically for > > initialization. If we later assign a new value into ptr, we will still be > > converting the new value to the type int (*)[< count * 10 >], using the > > value computed at the time of declaration of the variable. This model would > > simply require that conversion to validate the bounds during assignment just > > as it would during initialization. Yes, this is how it works in my local GCC. The same code instruments initialization, assignment, and arguments. > > Now, with nested arrays, variance does become a problem. Let’s reduce > > bounds expression to their evaluated bounds to make this easier to write. > > * int (*)[11] can be converted to int(*)[10] because we’re simply > > allowing fewer elements to be used. > > * By the same token, int (*(*)[11])[5] can be converted to > > int (*(*)[10])[5]. This is the same logic as the above, just with an > > element type that happens to be a pointer to array type. > > * But int (*(*)[11])[5] cannot be safely converted to int (*(*)[11])[4], > > because while it’s safe to read an int (*)[4] from this array, it’s > > not safe to assign one into it. > > * int (* const (*)[11])[5] can be safely converted to > > int (* const (*)[11])[4], but only if this dialect also enforces const- > > correctness, at least on array pointers. At the moment I do not allow implicit change of any bound in the type during assingment. For safe slicing the outermost array I currently wrap it in a macro: https://godbolt.org/z/n7xTfvnz3 > > Anyway, a lot of this changes if we want to use the same concept for > > non-local pointers to arrays, because we no longer have an obvious point of > > execution at which to evaluate the bounds expression. Instead, we are forced > > into re-evaluating it every time we access the variable holding the array. > > Consider: > > struct X { int count; int (*ptr)[count * 10]; // using my preferred syntax > > }; void test(struct X *xp) { // For the purposes of the conversion check > > here, the // source type is int (*)[< > > xp->count * 10 >], freshly // evaluated as part of the member access. int > > (*local)[100] = xp->ptr; } Yes, except for the syntax, this is how we plan to do it. Based on my experience with other languages, I also think it is useful to be able to refer to bounds outside of the struct definition, e.g. int foo(int n, struct foo { char (*ptr)[n]; } x); struct foo { char (*ptr)[10]; } x = ...; foo(10, x); which GCC happens to support already: https://godbolt.org/z/ja7qvMqPE This becomes useful if you want to be able to express the relation of different types, e.g. that two data structures used as input for something point to buffers of the same length. But the dependent sum type that we still miss is definitely more important. > > This has several immediate consequences. > > Firstly, we need to already be able to compute the correct bound when we do > > the dynamic checks for assignments into this field. For local variably- > > modified types, everything in the expression was already in scope and > > presumably initialized, so this wasn’t a problem. Here, we’re not helped > > by scope, and we are dependent on the count field already having been > > initialized. > > Secondly, we must be very concerned about anything that could change the > > result of this evaluation. So we cannot allow an arbitrary expression; > > it must be something that we can fully analyze for what could change it. > > And if refers to variables or fields (which it presumably always will), we > > must prevent assignments to those, or at least validate that any > > assignments aren’t causing unsound changes to the bound expression. > > Thirdly, that concern must apply non-locally: if we allow the address of the > > pointer field to be taken (which is totally fine in the local case!), > > we can no directly reason about mutations through that pointer, so we > > have to prevent changes to the bounds variables/fields while the pointer is > > outstanding. > > And finally, we must be able to recognize combinations of assignments, > > because when we’re initializing (or completely rewriting) this structure, > > we will need to able to assign to both count and ptr and not have the > > same restrictions in place that we would for separate assignments. Right, to make it safe you need to enforce such additional rules. > > None of this falls out naturally from separate, local language rules; it > > all has to be invented for the purpose of serving this dynamic check. This is true, we see this already for "counted_by". Still, even without enforcement of additional rules, you have some benefits: - The bound is visibly connected to the pointer. Currently, it is not and one has to guess. - The access (or other operations) via the pointer can be bounds-checked, assuming the bound is set correctly - The bound can be read via sizeof in other code, which prevents errors. - You can use it to build safer abstractions around it (For example I am experimenting with a vector type and you can access its content as an array that is bounds-checked in this way) Then in some opt-in strict mode we could enforce additional rules that make it perfectly safe. > > And in fact, -fbounds-safety has to do all of this already just to make > > basic checks involving pointers in structs work. I assumed it has to do similar checks anyway. > > If that can all be established, though, I think the type-conversion-based > > approach using variably-modified types has some very nice properties as a > > complement to what we’re doing in -fbounds-safety. I think we are in argreement here. > > For one, it interacts with the -fbounds-safety analysis very cleanly. If > > bounds in types are dynamically enforced (which is not true in normal C, > > but could be in this dialect), then the type becomes a source for reliable > > reliable information for the bounds-safety analysis. > > Conversely, if > > a pointer is converted to a variably-modified type, the analysis done > > by -bounds-safety could be used as an input to the conversion check. > > For another, I think it may lead towards an cleaner story for arrays of > > pointers to arrays than -fbounds-safety can achieve today, as long as > > the inner arrays are of uniform length. > > But ultimately, I think it’s still at best a complement to the attributes > > we need for -fbounds-safety. I agree we need to have the attributes, if just for annotating legacy APIs where you can not change the types. Martin > >
