On Mon, 28 Sep 2026 00:48:39 GMT, Ashay Rane <[email protected]> wrote:
>> Prior to this patch, the `StackYellowPages` and `StackRedPages` >> influenced the total reserved stack memory, but they didn't change the >> minimum stack size specified as required by HotSpot to Windows. This >> patch adds the call to `SetThreadStackGuarantee()` to set the minimum >> stack size. >> >> The key change here is that the argument to `SetThreadStackGuarantee()` >> is set to one less than the total number of yellow pages. The one less >> page is because Windows uses a guard page (access to which tells Windows >> to commit more stack pages); in the terminal case, we want the guard >> page to coincide with the top-most yellow stack page, thus leaving all >> except one yellow page available as the stack size. >> >> Importantly, however, we do not use the red page count as the argument >> to `SetThreadStackGurantee()`, since the red stack pages are unavailable >> for handling recoverable overflows. The red page count impacts the >> _total reserved_ stack size, just not the _minimum_ stack size. Still, >> forcing the red stack page count to be included into the computation of >> the argument to `SetThreadStackGurantee()` causes HotSpot to fail with >> the assertion `assert(!in_vm) failed: Undersized StackShadowPages`, >> since Windows is unable to commit more stack pages due to the minimum >> stack size now being larger than just the yellow page count minus one. >> >> The accompanying test passes zero to `SetThreadStackGuarantee()` to >> probe the current minimum stack size, which we then compare against the >> expected size based on the yellow page count. The same test fails >> without this patch on both Windows/x64 and on Windows/ARM64. >> >> Validated this patch by running through all tier 1, 2, and 3 HotSpot >> jtreg tests on Windows/x64 and Windows/ARM64 in FastDebug config. >> >> --------- >> - [x] I confirm that I make this contribution in accordance with the >> [OpenJDK Interim AI Policy](https://openjdk.org/legal/ai). > > Ashay Rane has updated the pull request incrementally with one additional > commit since the last revision: > > Handle EXCEPTION_STACK_OVERFLOW when yellow zone is unguarded > > The previous code did not handle a corner case that arises when HotSpot > receives `EXCEPTION_STACK_OVERFLOW` (since the guard page cannot be > moved further down) while already processing a recoverable overflow. In > this case, the yellow and reserved zones are writable (as part of the > recoverable overflow), so we retry the same instruction. If the > faulting probe reaches the red zone, we'll receive > `EXCEPTION_ACCESS_VIOLATION`, which will be handled as an unrecoverable > overflow. I can't tell from your example which pages are using PAGE_GUARD, and what protection the reserved pages 4 and 5 are using. While Windows reports the guarantee area is 2 pages, it looks like it is effectively 3 pages total (pages 1 through 3 in your example). If we set the HotSpot red zone to this size, then all the yellow pages should be outside of the guaranteed zone and cause EXCEPTION_ACCESS_VIOLATION. Only a fault in the red zone would cause EXCEPTION_STACK_OVERFLOW, which we could treat as fatal. If that works, wouldn't that simplify things? I hope to get access to a Windows machine soon to experiment with this myself. ------------- PR Comment: https://git.openjdk.org/jdk/pull/32365#issuecomment-5902040422
