> The way our mmap() hinting works is that we record the address of the > last successful mmap() and then base the hint for the next mmap() on > that previously-recorded address, adjusted appropriately upwards or > downwards for required length and alignment. The idea is that a > use-after-free bug should then ideally manifest as a SIGSEGV rather > than reading random garbage from a subsequent, unrelated mmap().
That does not make sense to me. Even our libc malloc(3) attempts at finding such probelms FAR FAR EXCEEDS the minimal effort you are trying to do here. Maybe you should go look at our libc malloc.c for ideas that actually find application bugs on a regular basis. Hint: Most overflow problems happen below the page boundary. > As you can imagine, if you do the above in a loop for long enough, > you'll eventually bump up against VM_MAXUSER_ADDRESS and get ENOMEM > due to the hint sitting within that region. I'll ask you again where you are getting VM_MAXUSER_ADDRESS from, and what it has to do with anything!! > The point here is that we actually *don't* want to have to know about > VM_MAXUSER_ADDRESS, nor have to retry mmap() without a hint when we > get ENOMEM; we would rather the kernel just ignore the hint if addr >= > VM_MAXUSER_ADDRESS, and place the mapping wherever it thinks is best. Why are you getting ENOMEM? Because you specified an impossible address. How did you calculate that address? Poorly. > But also, my question about the randomness of mmap(NULL, ...) earlier > still stands; if the resulting addresses are actually highly random, On OpenBSD, mmap(2) has been returning highly random addresses since 2003, and malloc(3) has been relying only on mmap(2) since 2008. > then the above scheme may just be a complete waste of time on OpenBSD > anyway, because the OS would already be doing a better job of it than > we could (which would be great!). Applications or compilers are the wrong place to fix a problem which the operating system call level should fix. > I still think the kernel behavior here is odd and should be changed to > be consistent one way or the other, but I guess it wouldn't affect us > anymore. I'm still trying to understand why you keep talking about VM_MAXUSER_ADDRESS. If you want a place where a hint will fail, you can try mmap(main, 1, ...
