[media_addon_server] crash after boot

[diver]

media_addon_server crashes when starting Haiku (~50% of boots).
I have BeStartup.wav set as startup sound.
Backtrace attached.

Tested in hrev35356 under VirtualBox 3.0.12

[atomozero]

bug on real hardware

[axeld]

atomozero: the bug you encountered is a problem of the short cut filter input_server add-on, and has nothing to do with this bug.

[diver]

Can't reproduce it on recent revisions, but startup sound doesn't play too.

[diver]

Just happened again, sorry for the noise.

[Disreali]

I've been experienced this a lot on recent revisions, 38200 or so and newer.

[Disreali]

On hrev38511, the media_addon crashed after 1hour 23 minutes. Attaching bt.

[Disreali]

This issue seemed to be gone for awhile, but is now back in hrev39917-2h and hrev40044-2h.

Sometimes, it crashes within minutes of booting up, other times it happens hours later. Either way, Haiku still does not play my assigned startup sound.

To remedy, I need to open the Media preflet, which now lists no audio hardware and press the "Restart Media Services" button and close pref. Occasionally this is all that is needed, though most times(75%) I need reopen Media preflet which displays an Alert box stating 'media services are down, would you like to restart?', press yes to restart and still after this I need to open media pref a third time to restart the media services before Sounds or MediaPlayer will play any audio and video files.

The backtrace is the same as one I previously attached.

Is there any other troubleshooting that I could do to help out?

[Disreali]

This issue still occurring on hrev42636-2h nightly image. I am unable to play any sounds or videos on this rev, regardless of how many times I restart Media Services.

[Disreali]

I am noticing that the media_addon_server tends to fail more often when there is greater system load, such as when I am building Haiku.

I also noticed that the server gradually eats more and more memory until it fails or I kill it and restart the Media Services. It starts out using about 2MB but increases from there, according to ProcessController's Memory Usage view. I have seen it get up to about 55MB after several hours of uptime without any playing of sounds or media. Once restarted the medai_addon-sever returns to using about 2MB and again starts to eat more memory until the next failure which is usually several hours after restart.

Attaching backtrace from crash on hrev42637-2h nightly and screen captures of memory usage as shown by ProcessController.

[Disreali]

Is this issue possibly related to ticket:5863?

[Disreali]

Attaching a Syslog with stack-crawl of media_addon_server from a hrev43005-gcc2hybrid nightly install.

[marcusoverhagen]

Replying to Disreali:

Attaching a Syslog with stack-crawl of media_addon_server from a hrev43005-gcc2hybrid nightly install.

This is memory corruption.

2011-11-02 01:05:52 KERN: 26823: DEBUGGER: oops
2011-11-02 01:05:52 KERN: debug_server: Thread 26823 entered the debugger: Debugger call: `oops'
2011-11-02 01:05:52 KERN: stack trace, current PC 0xffff0114  :
2011-11-02 01:05:52 KERN:   (0x78174bac)  0x50a18a  SetHeader__7BBufferPC12media_header + 0x32


2011-11-02 17:43:21 KERN: 261: DEBUGGER: oops
2011-11-02 17:43:21 KERN: debug_server: Thread 261 entered the debugger: Debugger call: `oops'
2011-11-02 17:43:21 KERN: stack trace, current PC 0xffff0114  :
2011-11-02 17:43:21 KERN:   (0x78174bac)  0x50a18a  SetHeader__7BBufferPC12media_header + 0x32
2011-11-02 17:43:21 KERN:   (0x78174bdc)  0x50b1fe  HandleMessage__15BBufferConsumerlPCvUl + 0x1ae

      case CONSUMER_BUFFER_RECEIVED:
	        {
	            const consumer_buffer_received_command* command
	                = static_cast<const consumer_buffer_received_command*>(data);
	
	            BBuffer* buffer = fBufferCache->GetBuffer(command->buffer);
	            buffer->SetHeader(&command->header);

BBuffer::SetHeader(const media_header* header)
{
    CALLED();
    ASSERT(header->buffer == fMediaHeader.buffer);
    if (header->buffer != fMediaHeader.buffer)
        debugger("oops");
    fMediaHeader = *header;
}

[Disreali]

Replying to marcusoverhagen:

This is memory corruption. It's not a crash either, but a debugger call.

I chose to do the gdb debug before dropping to KDL. That may have been the wrong choice. I'll be sure to do the KDL stack traces first next time.

[Disreali]

I brought the system out of the screensaver to see that the server crashed all by itself while the saver was running.

From the syslog:

2011-12-29 12:41:00 KERN: 254: DEBUGGER: oops
2011-12-29 12:41:00 KERN: debug_server: Thread 254 entered the debugger: Debugger call: `oops'
2011-12-29 12:41:01 KERN: stack trace, current PC 0xffff0114  :
2011-12-29 12:41:01 KERN:   (0x78174b9c)  0x51618a  SetHeader__7BBufferPC12media_header + 0x32
2011-12-29 12:41:01 KERN:   (0x78174bcc)  0x5171fe  HandleMessage__15BBufferConsumerlPCvUl + 0x1ae
2011-12-29 12:41:01 KERN:   (0x78174e4c)  0x5271df  WaitForMessage__10BMediaNodexUlPv + 0x1df
2011-12-29 12:41:01 KERN:   (0x78178e8c)  0x52178b  ControlLoop__17BMediaEventLooper + 0x18f
2011-12-29 12:41:01 KERN:   (0x78178f6c)  0x521c2f  _ControlThreadStart__17BMediaEventLooperPv + 0x37
2011-12-29 12:41:01 KERN:   (0x78178fac)  0x60ae27  thread_entry + 0x23

Attaching gdb backtrace.

[ttcoder]

The debugger("oops") issue (it is indeed this ticket that tracks it, right?) has taken a sharp turn for the worse for me.. Appeared on this laptop last week, but I could get away with a reboot.. But with today's update to hrev48168 I can't possibly play much media at all without crashing media_addon_server with the "oops" tripwire.. (or sometimes the segmentation fault in float_to_float, making me think that maybe it's the mixer that's corrupting something in the address space of the server).

Before getting into more involved stuff, a question.. I once got the "low resource" warning in the syslog right before the crash, playing an mp3 a few seconds after boot-up when obviously there were no resources being wasted:

DAEMON 'DHCP': /dev/net/broadcom570x/0: DHCP status = No error
KERN: low resource memory: normal -> critical
KERN: low resource memory: critical -> normal
 ..
KERN: 605: DEBUGGER: oops
KERN: debug_server: Thread 605 entered the debugger: Debugger call: `oops'

So my question is.. What can trigger a low resource memory: normal -> critical warning, apart from genuine resource waste?

An add-on going into an infinite loop doing malloc() ? An add-on going in an infinite loop reading "wild" memory outside of its own memory bounds ?

Bonus question -- are there open Coverity CIDs on MediaAddonServer.cpp and friends ?

Bonus²: sometimes the syslog contains this error:

KERN: runtime_loader: /boot/system/lib/x86/libavcodec.so.54.23.100: Failed to get gcc version.
KERN: runtime_loader: /boot/system/lib/x86/libavutil.so.51.54.100: Failed to get gcc version.

But it seems to happen in the context of the float_to_float() crash, so off topic for this ticket probably.

[Barrett]

Fixed in hrev49503.