non-debggable app_server crash

[pulkomandy]

This has happened randomly forsome time while running the webkit testsuite. Now it happens reliably. Symptoms: screen just stops drawing. Attaching to the machine by ssh I noticed app_server is not running anymore. And I see this in the syslog:

KERN: 100610: DEBUGGER: Could not create BWindow's receive port, used for interacting with the app_server!
KERN: _user_debugger(): Failed to install debugger. Message is: `Could not create BWindow's receive port, used for interacting with the app_server!'
KERN: thread_hit_debug_event(): Failed to create debug port: No more ports available
KERN: 112390: DEBUGGER: Could not create BWindow's receive port, used for interacting with the app_server!
KERN: _user_debugger(): Failed to install debugger. Message is: `Could not create BWindow's receive port, used for interacting with the app_server!'
KERN: thread_hit_debug_event(): Failed to create debug port: No more ports available
KERN: vm_soft_fault: va 0x0 not covered by area in address space
KERN: vm_page_fault: vm_soft_fault returned error 'Bad address' on fault at 0x3e8, ip 0xa9b7ca, write 0, user 1, thread 0x1aa35
KERN: vm_page_fault: thread "a:109077:x-vnd.DumpRenderTree" (109109) in team "app_server" (958) tried to read address 0x3e8, ip 0xa9b7ca ("app_server_seg0ro" +0xa37ca)
KERN: thread_hit_serious_debug_event(): Failed to install debugger: thread: 109109: No more ports available

It would be nice to have some better user feedback in that case (panic or a message to /dev/console). I will try to find why I seem to be running out of ports.

[anevilyak]

AFAICT this didn't even get far enough to try to load/attach Debugger.

[bonefish]

Apparently the system is out of ports. Since the debugging infrastructure is based on port communication, it fails as well. So the actual question is who is leaking ports in the first place.

[pulkomandy]

Yes, I know that, and I'm trying to debug it - not easy with the Debugger not running :)

The port leak is because of webkit leaking BBitmaps which accept views.

The problem is the user experience if this happens. The app_server crashes and there is no on-screen error message or anything. I think the system should panic or at least send some error message to /dev/console to let the user know what happened. But leaving the system with a frozen screen and no error message is not nice.

[diver]

#8372 is possibly related.

[ttcoder]

Seems to be still there in hrev49480 : the "freeze, and debugger could not launch" part just happened to me after an app_server crash; not sure if the app_server crash itself is due to running out of ports though. I had W+ and qupzilla running for an hour or more:

KERN: vm_soft_fault: va 0x0 not covered by area in address space
KERN: vm_page_fault: vm_soft_fault returned error 'Bad address' on fault at 0x3e8, ip 0x2e862e, write 0, user 1, thread 0x269
KERN: vm_page_fault: thread "a:615:x-vnd.QupZilla" (617) in team "app_server" (377) tried to read address 0x3e8, ip 0x2e862e ("app_server_seg0ro" +0xa462e)
KERN: thread_hit_serious_debug_event(): Failed to install debugger: thread: 617: No more ports available
KERN: Keyboard Requested Halt.
KERN: Welcome to Kernel Debugging Land...

[bonefish]

To make progress with this ticket, it might be good idea for the debug server to enter the kernel debugger when installing the debugger on a crashed critical server fails. That would at least allow having a look at the situation in the kernel debugger. I guess looking at the ports would already help a lot.