ASSERT FAILED: newCommitment <= (topCache->virtual_end - topCache->virtual_base)

[waddlesplash]

While experimenting with gnuplot, Xlibe, and the guarded heap, I got this KDL:

PANIC: ASSERT FAILED (../src/system/kernel/vm/vm.cpp:6991): newCommitment <= (topCache->virtual_end - topCache->virtual_base)
Welcome to Kernel Debugging Land...
Thread 65085 "gnuplot" running on CPU 2
stack trace for thread 65085 "gnuplot"
    kernel stack: 0xffffffff81157000 to 0xffffffff8115c000
      user stack: 0x00007f45695c0000 to 0x00007f456a5c0000
frame                       caller             <image>:function + offset
 0 ffffffff8115bb70 (+  16) ffffffff802723ce   <kernel_x86_64> arch_debug_stack_trace + 0x13
 1 ffffffff8115bb90 (+  32) ffffffff801a1b74   <kernel_x86_64> stack_trace_trampoline(void*) + 0x11
 2 ffffffff8115bbb0 (+  32) ffffffff802660c0   <kernel_x86_64> arch_debug_call_with_fault_handler + 0x1a
 3 ffffffff8115bc00 (+  80) ffffffff801a3e35   <kernel_x86_64> debug_call_with_fault_handler + 0x7a
 4 ffffffff8115bc90 (+ 144) ffffffff801a1e6b   <kernel_x86_64> kernel_debugger_loop(char const*, char const*, __va_list_tag*, int) + 0x2f4
 5 ffffffff8115bcd0 (+  64) ffffffff801a22ca   <kernel_x86_64> kernel_debugger_internal(char const*, char const*, __va_list_tag*, int) + 0x76
 6 ffffffff8115bdc0 (+ 240) ffffffff801a4477   <kernel_x86_64> panic + 0xc9
 7 ffffffff8115bf20 (+ 352) ffffffff80249215   <kernel_x86_64> _user_set_memory_protection + 0x5e5
 8 ffffffff8115bf30 (+  16) ffffffff80267d7f   <kernel_x86_64> x86_64_syscall_entry + 0xfb
user iframe at 0xffffffff8115bf30 (end = 0xffffffff8115bff8)
 rax 0xd6                  rbx 0xae332a51f0          rcx 0x17cc2d2dccc
 rdx 0x3                   rsi 0x1000                rdi 0xae3342c000
 rbp 0x7f456a5bf8e0         r8 0xae3327a048           r9 0xae3327a060
 r10 0xae332a53f0          r11 0x202                 r12 0xae3327a000
 r13 0x176000              r14 0x0                   r15 0xae332a51f0
 rip 0x17cc2d2dccc         rsp 0x7f456a5bf8c8     rflags 0x202
 vector: 0x63, error code: 0x0
 9 00007f456a5bf8e0 (+   0) 0000017cc2d2dccc   <libroot.so> _kern_set_memory_protection + 0x0c

[waddlesplash]

What appears to be happening here is that the area in question was created with READ|WRITE protection, and thus was committed in full. Then later set_memory_protection was called, at which point a number of pages were switched from READ|WRITE to just READ (or no protection), but the commitment was not downsized appropriately.

area->name = "guarded_heap_area"

topCache->virtual_base = 0
topCache->virtual_end = 2097152
topCache->committed_size = 2097152

becomesWritable = true
commitmentChange = 4096
newCommitment = 2101248

I'm not sure how this could've happened, since when the area_page_protections array is allocated it defaults to having the area's own protections, and then the commitment will be adjusted appropriately after that.

It appears the topCache does have a source cache, which is interesting. The source cache does not have any areas, but does have two consumers. That may explain how we've gotten into this state.

[X512]

100% reproducible for me with Wine 9.0+ with Wayland backend.

[waddlesplash]

I tried to come up with a reproducer in the tests I added in hrev58197, but it doesn't trigger this for me.

[X512]

Also observed with Firefox: ​https://www.youtube.com/watch?v=wH25VoTC_iQ .