Media Kit Test Baseline

[kallisti5]

Xiph has a nice selection of open test videos here: ​https://media.xiph.org/video/derf/

It would be good to form a baseline based on these videos to track improvements / regressions to our video decoding.

[pulkomandy]

These are uncompressed videos, so they test only the most boring parts of decoding (in a format we probably can't even decode yet?).

There is already a large testsuite of many common and uncommon formats (and various broken files) at ​http://samples.ffmpeg.org (and a mirror at mplayer).

There is also a set of freely redistributable videos in our "sample" packages.

What we need is not a set of videos, but an automated way to test them: 1) No crashes when decoding 2) Correct framerate and infos in BMediaTrack/BMediaFile 3) That decoding is not complete crap (I don't know how to test this? Maybe compare a few select frames with PNG extracted using ffmpeg directly?)

[cocobean]

I've tested the mainstream videos on Haiku (currently testing hrev52012). Many of them now work using Mplayer and MediaPlayer. I'd suggest recompiling MediaPlayer using the current decoders/libvpx(?)/FFMPEG 3.4.2 backend and improve buffering test. I test at either 1080p or 4K (4000x2250) videos for stress testing on Haiku. So far, it is good for most videos. Things may differ on the x86-side due to memory handling of large files/buffering/disk/etc. If only this was embedded...

I usually suggest pro-grade encoding beforehand for very good samples or 'tested' codecs compiled on Haiku. I've collected many over the years or test what people post so I can help test this area further for you. The websites have some good ones and your BeScreenCapture works for me as well. Again, you probably want to make sure the included media apps on the nightly images are all recompiled against recent codecs...