Opened 5 years ago

Closed 4 years ago

#15263 closed bug (fixed)

Media_addon_server crash when attempting to reboot or shut down Haiku (Copy2Int())

Reported by: vidrep Owned by: leavengood
Priority: normal Milestone: R1/beta2
Component: Servers/media_addon_server Version: R1/Development
Keywords: Cc: jerome.duval@…, ttcoder
Blocked By: Blocking: #15260, #15332, #15530, #15676
Platform: All

Description

hrev53379 x86_64

Since updating to hrev53379 I am experiencing a crash of the media_addon_server whenever I reboot or shutdown Haiku.

On the third attempt I was able to generate a debug report (attached)

Attachments (9)

media_addon_server-6818-debug-17-08-2019-00-17-43.report (14.2 KB ) - added by vidrep 5 years ago.
media_addon_server-4693-debug-19-08-2019-03-33-01.report (13.8 KB ) - added by vidrep 5 years ago.
syslog.txt (24.8 KB ) - added by vidrep 5 years ago.
media_addon_server-471-debug-06-10-2019-13-02-36.report (12.5 KB ) - added by vidrep 4 years ago.
media_addon_server-563-debug-24-11-2019-21-45-04.report.txt (15.9 KB ) - added by dsuden 4 years ago.
media_addon_server crash at restart - latest Haiku 64 bit version
media_addon_server-494-debug-22-12-2019-12-11-18.report (1.2 KB ) - added by nephele 4 years ago.
media_addon_server-494-debug-22-12-2019-12-16-57.report (12.8 KB ) - added by nephele 4 years ago.
longer report hrev53644
log.txt (3.2 KB ) - added by vidrep 4 years ago.
log.2.txt (6.9 KB ) - added by nephele 4 years ago.
media_server log logg

Download all attachments as: .zip

Change History (47)

comment:1 by vidrep, 5 years ago

Owner: changed from nobody to leavengood
Status: newassigned

comment:2 by waddlesplash, 5 years ago

Blocking: 15260 added

comment:3 by nephele, 5 years ago

This seams to be easily reproducible by asking media_addon_server to quit from the team manager, this gives me a quite similar debug report (i can attach this if that is relevant too)

same hrev for me.

comment:4 by vidrep, 5 years ago

It seems to only happen after I have been playing a media file. If I have not been playing any media during a session, then Haiku will shut down cleanly.

Version 0, edited 5 years ago by vidrep (next)

comment:5 by nephele, 5 years ago

Wierd, with asking it to quit from the team monitor the behaviour seams oposite to that, i have had it crash on shutdown without having used any media stuff in a session, if i quit it (and it dies) and subsequently try to play something the media server will be restarted (upon request) and quiting media_addon_server after that doesn't produce a crash anymore.

comment:6 by X512, 5 years ago

It probably caused by improper cleanup when media_addon_server quit. Crash occured in MultiAudioNode thread which is still active when some dependent resource is released. media_addon_server should stop and delete all nodes it hosts before quit.

Last edited 5 years ago by X512 (previous) (diff)

comment:7 by vidrep, 5 years ago

Had it happen again. I've attached the new debug report and syslog.

by vidrep, 5 years ago

Attachment: syslog.txt added

comment:8 by leavengood, 5 years ago

I will look at this within the next few days. I am almost done with my current Tracker fix.

I think X512 has the right idea here, though it seems like a pretty basic thing that the media_addon_server would stop all its nodes before quitting...

I just did some quick testing and I cannot seem to reproduce this, but I might not be using the right version of media_addon_server. I am also testing in VirtualBox and whatever version of Intel HD they have doesn't seem to work, and this crash seems to be dependent on that (multi_audio media add-on.) And this machine has a Realtek so I also do not think I would see this if I booted natively. But I'll keep trying.

comment:9 by leavengood, 5 years ago

Cc: jerome.duval@… added

Jérôme, you may know this code better than I, any insights? One question I have is why this was commented out: https://xref.plausible.coop/source/xref/haiku/src/add-ons/media/media-add-ons/multi_audio/MultiAudioNode.cpp#1294

We suspect this thread is still trying to access buffers after they have been deleted from the BufferCache during shutdown due to my change in hrev53379.

comment:10 by X512, 5 years ago

Trying to build media_addon_server with debug info by using "DEBUG=1 jam -q media_addon_server", but release build is still produced. media_server debug build complete without problems. What I am doing wrong?

comment:11 by X512, 5 years ago

I look like problem is in multi_audio add-on. media_addon_server termination sequence seems to be correct. Crash occured when mixer node are released by media_addon_server in _DestroyInstantiatedFlavors. mixer is released before MultiAudioNode, so MultiAudioNode::_OutputThread is still running. MultiAudioNode don't correctly respond to stop and disconnect notifications. Same crash can be also caused by removing connection between "Audio Mixer" and "HD Audio" nodes.

Last edited 5 years ago by X512 (previous) (diff)

comment:12 by leavengood, 5 years ago

Thanks for digging into this X512. We have some other media-related bugs about not being able to change the media output without having to reboot afterwards, so I think there is some refactoring that is needed here to decouple the mixer from multi audio. I haven't gotten to that part of the media kit code yet so I still have some things to learn.

comment:13 by diver, 5 years ago

Blocking: 15332 added

comment:14 by vidrep, 4 years ago

Still here on hrev53528_x86_gcc2. Debug report attached.

comment:15 by dsuden, 4 years ago

I can confirm this one, am seeing it pretty consistently in latest Haiku 64 bit builds...kind of an "ouch" issue because it hurts credibility a little. I'll attach a report.

by dsuden, 4 years ago

media_addon_server crash at restart - latest Haiku 64 bit version

comment:16 by ttcoder, 4 years ago

Cc: ttcoder added
Summary: Media_addon_server crash when attempting to reboot or shut down HaikuMedia_addon_server crash when attempting to reboot or shut down Haiku (Copy2Int())

@Dane you could configure debug_server to silently drop media_addon_server when it crashes, much like we did with CC6 quietly-generate-report thing. That's a hackish work-around at best, but then again media_addon_server has been doing that kind of thing for me ever since I started using Haiku in the alpha days (10+ years ago), so expecting a quick resolution might be less reasonable than applying a work-around here :-)

comment:17 by jmairboeck, 4 years ago

Now that my sound card works with the patched HDA driver (see #15499) I see media_addon_server crashing on shutdown or reboot. Before that fix I did not see the crash. Maybe this helps in finding the cause for this.

comment:18 by humdinger, 4 years ago

Blocking: 15530 added

comment:19 by waddlesplash, 4 years ago

Resolution: fixed
Status: assignedclosed

Fix merged in hrev53644.

comment:20 by nephele, 4 years ago

I had another crash on shutdown with media_addon_server on hrev53644, although i am unsure of whether it is the same issue or a different one.

comment:21 by diver, 4 years ago

Resolution: fixed
Status: closedreopened

comment:22 by korli, 4 years ago

Looks like a timing issue then. I can't reproduce anymore at least.

comment:23 by X512, 4 years ago

Still reproduced on hrev53644 x86_64.

Steps to reproduce:

  1. Play some sound.
  2. Exit media server by ProcessController -> Quit an Application.
  3. media_addon_server will crash.

100% reproducible.

comment:24 by X512, 4 years ago

Older version (https://git.haiku-os.org/haiku/tree/src/add-ons/media/media-add-ons/multi_audio/MultiAudioNode.cpp?id=be4e010170a900c3c38b00e17f6104a7bdb938ed) used fBuffer_free semaphore that was deleted later. Maybe it was actually required.

comment:26 by X512, 4 years ago

Disconnecting HD Audio from Audio Mixer in Cortex also do not produce crash after fix.

comment:27 by korli, 4 years ago

@nephele could you do the following?

  • in a Terminal: launch_roster stop x-vnd.haiku-media_server
  • then quit media_server with ProcessController
  • in Terminal: /system/servers/media_server
  • then quit media_server with ProcessController

Please post the log.

comment:28 by vidrep, 4 years ago

Just updated to hrev53644 and ran a couple of tests. The crash only occurs when a media file with an audio component is played first (see comment:4).

comment:29 by korli, 4 years ago

@Vidrep you could also try the steps of comment:27

comment:30 by vidrep, 4 years ago

korli, log.txt attached.

by vidrep, 4 years ago

Attachment: log.txt added

by nephele, 4 years ago

Attachment: log.2.txt added

media_server log logg

comment:31 by nephele, 4 years ago

korli: i have done what you asked, however i could not reproduce a crash this way :/

in reply to:  31 comment:32 by korli, 4 years ago

Replying to nephele:

korli: i have done what you asked, however i could not reproduce a crash this way :/

Maybe try to log to a file "/system/servers/media_server 2>&1 1>media_server.log"

comment:33 by nephele, 4 years ago

How does redirecting file descriptors help? it just did not crash with that test :), i have already added the full log of the test (and media_server) to the ticket

comment:34 by vidrep, 4 years ago

In my case it did crash. Do you need the debug report in addition to the terminal log?

comment:35 by Parnikkapore, 4 years ago

I was playing around with this today, and apparently, if you kill the mixer thread ("Yeah baby, very shagadelic") manually first, media_addon_server will not crash on shutdown (Yesterday's hrev)?

Might give a hint as to what's wrong...

comment:36 by diver, 4 years ago

Blocking: 15676 added

comment:37 by pulkomandy, 4 years ago

Milestone: UnscheduledR1/beta2

comment:38 by pulkomandy, 4 years ago

Resolution: fixed
Status: reopenedclosed

Fixed in Fixed in hrev53876.

Note: See TracTickets for help on using tickets.