Assist video decoding with multiple threads

[PieterPanman]

My experiences from some hd videoclips shows that we're not taking advantage of multiple cores for heavy video decoding. Since I have a nice dual core system it is a shame it cannot play some videos smoothly. It might be a while before we get video acceleration from GPU's, so maybe video rendering can distribute frames across multiple threads. System is a Core2Duo laptop 2.0 GHz.

From ​http://www.microsoft.com/windows/windowsmedia/musicandvideo/hdvideo/contentshowcase.aspx : Coral_Reef_Adventure_1080.wmv Very stuttery, no sound. FrameGenerator thread full CPU (as expected I guess)

I Am Legend - Trailer.mp4 (1080p, h264) ​http://www.h264info.com/clips.html Extremely choppy, corrupted images, no sound. More problems but (see also #4511)

The Simpsons Movie - Trailer.mp4 720p h264 Plays ok, slightly choppy. Still causes mouse pauses every 30 seconds for 1 or 2 seconds. Sometimes it loses some frames and seems to have skipped a key frame, after which you can really see how compression works :) Can also crash mediaplayer, as with #4511

[PieterPanman]

Forgot to mention: ​http://dev.haiku-os.org/attachment/ticket/4511/lspci.txt and hrev33137.

[dlmcpaul]

We are currently using ffmpeg for h264 decoding. ffmpeg is not multithreaded so there is not much we can do here.

When playing a file the audio, video and extractor all run in seperate threads.

[PieterPanman]

Thanks for your reply. Sure, I appreciate that, no use in reinventing the wheel here. But, I was wondering, is there no way to instruct two ffmpeg instances to render odd and even frames? Or perhaps each 10 frames? Maybe take it to the Media Kit level, instead of the codec level so that each media file is instantly multi threaded? Probably each frame is too related to the previous, but I have to ask anyway, I think it sounds like a cool idea :).

[dlmcpaul]

I am not sure that is possible. each frame in H264 relies on both previous and future frames. If you could break out sets of frames that are independant then you could run more than 1 render thread but this really has to be done at the decoder level.

Maybe a decoder could multithread each frame. You might be able to break up a frame into independant elements and process each in a seperate thread.

A couple of questions. Do you know if you are using the vesa driver? ls /dev/graphics should show the video driver used.

If you are. Drop me a line at dlmcpaul .AT. gmail.com and you can try a test ffmpeg version that does the yuv2rgb conversion faster. It might help on h264 video that is borderline for your machine.

[PieterPanman]

Yes, I am using the vesa driver, as the Geforce 8xxx series is not yet supported by Rudolf's Nvidia driver. Thanks for the offer, check your email.

Regards, Pieter

[stippi]

There is an "ffmpeg-mt" branch which seems to be in active in development, at least a couple weeks ago when I last looked. This is probably very promising. At the MediaKit level, it is not possible for the reasons you have already guessed (interdependence of frames). h.264 is even more tricky since frames can rely on pictures outside of the current group of pictures. And groups of pictures can span quite a lot of frames, certainly more than 100. So if GOPs where to be decoded independently, it would introduce a huge latency to the playback. I think ffmpeg-mt is our best hope, but it has to mature first.

[PieterPanman]

Promising indeed, I just looked at it. Saw in a post in a forum that someone gained almost 350% increase in frames per second by going from 1 thread to 4, that was in februari. Scales very well, then. ​http://ubuntuforums.org/showthread.php?t=1049449 Can't wait, perfect for Haiku.

[korli]

hrev44080 now gets multithread decoding for ffmpeg.