MediaConverter doesn't convert to OGG or MP3 and converts to WAV with video

[Kev]

I tried converting some MP4 videos to audio-only ogg vorbis or MPEG w/ AC-3 (same as MP3?). Every file I tried resulted in a small file (only a few kb) quickly created.

Raw audio in a WAV container worked; however, despite "no video" in the video encoding section, the WAV files, when opened in MediaPlayer, show the original video somehow. (I didn't even know WAV could do that.)

The MP4s in question play fine in MediaPlayer, which also plays other OGGs just fine.

hrev45413 gcc4hybrid. AC '97 Audio, Dolby 5.1 (6-channel)

[Kev]

As a side note, the WAVs are "normal" enough that BeOS' CDBurner app successfully burns audio CDs with them.

[ttcoder]

ping (hrev47850)

I too have a use-case for converting a wav file to something more compact, like Ogg/Vorbis -- I'd like to encode with the media-kit instead of always relying on /bin/lame :-)

In the latest hrev I get this when trying to generate an Ogg/vorbis from a wav file:

.../New folder> MediaConverter 
[libvorbis @ 0x187e2038] Specified sample_fmt is not supported.
[ogg @ 0x187e6738] No extradata present
[libvorbis @ 0x187e2440] Specified sample_fmt is not supported.
Error writing audio frames: Initialization failed

and MediaConverter then crashes in converter_thread.

Augustin said now is the right time to resurface that kind of media-kit/ffmpeg issue, so there =)

(PS @Kev I think AC3 is lossless compression, different beast from mpeg/layer3)

EDIT: Should mention that I didn't try a gcc4 image of Haiku as it apparently wouldn't make a difference over the gcc2h I'm using.

EDIT2: heard that mpeg/layer3 encoding is not available in Haiku for patents reasons, so Ogg/Vorbis is the alternative that comes to mind

[ttcoder]

Might as well schedule for alpha5.. Quite a few people will want to not only read (decode) media but also write/encode it =)

[ttcoder]

... ping..! (in case it's relevant to the 3-changeset in hrev48039)

[pulkomandy]

It's relevant, but does not really fix. What I did is add FLAC (which works) and remove all other formats which are crashing. FLAC is a lossless codec which means it doesn't compress sound as well as Vorbis or MP3. Unfortunately I'm not sure there is a way to fix this unless we revert to an older ffmpeg - probably from alpha3 days. And this would remove support for decoding some modern video formats (h264 and the like). The other solution is to write an encoder using libvorbis directly (or musepack, or APE or whatever format we decide to support).

Moreover, after fixing this I found there is another bug and some of the files I tested with gets one to two seconds truncated at the end. I have not debugged this part, yet, but MediaPlayer manage to decode the original files until the end so there must be a way to avoid the issue.

[ttcoder]

With the crash fixed, this probably becomes an enhancement ticket.. Update on my side:

Upgraded to 48061 and indeed PCM, Vorbis ..etc are gone, and flac works as you describe Adrien.

ffmpeg: is the bug specific to Haiku (i.e. it fails to encode because it's compiled with our old gcc2) ? Or maybe it's a general bug in ffmpeg, but in that case we could file a ticket with them, telling them it's very good at decoding but fails to encode on all platforms, including Haiku ? I guess I like the idea of getting a half dozen encoders 'for free' in the Media Kit :-) but I'm confused about what the situation is with ffmpeg/encoders in Haiku exactly.

libvorbis: in case ffmpeg is really a non-starter and we have to create encoders separate from it, a Vorbis encoder would be my favorite as closest to mp3 compression probably.. I did a bit of work on haiku Decoders some years back, I guess doing an Encoder might be in my reach.. And if my work is good enough to submit back to trunk that'd be a way for me to give something back to the haiku project.. If you confirm this is the way to go I'll try it out and create a separate ticket (to not spam Kev) to report progress if any.

[pulkomandy]

I didn't investigate the ffmpeg problem deeply, maybe it's possible to debug it. Here is what I know about it:

• Because of our constraint to use gcc2, we are using an old version of ffmpeg. Currently we use version 0.10.14. ffmpeg current releases are at 2.something. They still have maintenance releases for the 0.10 branch, however (and also for the 0.9 branch. Older ones have no updates since 2013).
• Newer version (0.11 and later) won't compile at all with gcc2.
• Even 0.10 is known to not go along well with gcc2. We got it to compile with some patching, but it warns on the console that it has been miscompiled because the compiler didn't align stack variables. Even with this known issue, decoding seems to work fine.
• But I'm not sure the stack alignment issue is actually related to the problem. We compile ffmpeg without SSE2 and that should make it safe. It's easy to check this by testing MediaConvert on a gcc4 system and seeing if it also crashes there (after reenabling the vorbis encoder).

I think the best way forward is:

• First, we need to get MediaConvert to work reliably with FLAC. The problems I've hit in my testing were on the decoding side of things. It seems MediaConvert manages to decode most of the file, but fails at the last "frame" of audio, which is shorter than others because it hits the end of the file.
• Once we have this working, we can see if it helps with some of the other formats.
• If it doesn't, having a vorbis encoder using libvorbis directly is worth a try.

Do you know if using ffmpeg to encode to ogg/vorbis from the command line works? If it does, the problem is not related to gcc2 or the way we build ffmpeg, but to using it in a wrong way in our encoder plugin. Ant it should be possible to fix that.

[ttcoder]

Very clear picture of the situation, allowed me to dig into this. Not going to tackle the input decoding right now (naïve idea: maybe it's just mis-handling of B_LAST_BUFFER status code?), but focused on the encoders still.

It seems the fault is indeed with the gcc2 build of ffmpeg 0.x itself, independantly of the "glue" code that ties it into the Media Kit.

Indeed, using it stand-alone from the command-line I got nothing but errors, failures, and one crash (in malloc()) for these attempts:

ffmpeg -i 01breakin_benjamin__so_cold.mp3  01breakin_benjamin__so_cold.aac
ffmpeg -i 01breakin_benjamin__so_cold.mp3  01breakin_benjamin__so_cold.ac3
ffmpeg -i 01breakin_benjamin__so_cold.mp3  01breakin_benjamin__so_cold.alac
ffmpeg -i 01breakin_benjamin__so_cold.mp3  01breakin_benjamin__so_cold.ogg
ffmpeg -i 01breakin_benjamin__so_cold.mp3  01breakin_benjamin__so_cold.vorbis
ffmpeg -i 01breakin_benjamin__so_cold.mp3  -acodec vorbis 01breakin_benjamin__so_cold.ogg
ffmpeg -i 01breakin_benjamin__so_cold.mp3  -acodec vorbis -aq 100 01breakin_benjamin__so_cold.ogg

This one, however, is pure gold:

ffmpeg -i 01breakin_benjamin__so_cold.mp3  01breakin_benjamin__so_cold.mp2

Tried that encoding (mpeg layer 2) on a few files.. It outputs the same size and audio quality as lame, but where lame takes e.g. 11 seconds to create an .mp3, ffmpeg creates an .mp2 in less than 2 seconds (!). Awesome.

Question...

Could that be the base of a media-kit encoder, technically ? I guess so. As to the "lawyer" side of the equation, I know Haiku can't pay the $$ fee to enable this out of the box, but could it be made an optional build feature, so that some of us can just "jam -q {add-ons,add-ons/x86}/media/plugins/ffmpeg" and use it with the mp2 capability ?

Otherwise I'll ask about the feasibility of using the recent ffmpeg 2.x in a gcc4 or gcc4h build of Haiku :-) I suppose the best place to start would be to look for a recipe for those on Haikuports.

[pulkomandy]

Well, ffmpeg actually is built with support for this, so making it available through the media kit is not going to make the legal side any better or worse. It's just a matter of adding the format to the list (which is as easy as what I did for flac in ​http://cgit.haiku-os.org/haiku/commit/?id=hrev48039).

An ffmpeg 2 based add-on for gcc4 versions is also possible. There is already a recipe for ffmpeg 2.x but I'm not sure if it compiles currently. But it should be possible to port the changes from our previous ffmpeg ports. Then, the media plugin on Haiku side would need to be modified to use the new APIs from ffmpeg 2.x (it would probably beecome an "ffmpeg2" plugin and live next to the existing one).

Finally, another thing you can test:

pkgman install ffmpeg_x86
ffmpeg-x86 -i 01breakin_benjamin__so_cold.mp3  -acodec vorbis 01breakin_benjamin__so_cold.ogg

If that works, we will be sure our problem is compiling ffmpeg with gcc2. If the -x86 (gcc4) version of ffmpeg also crashes, the problem is elsewhere.

If the problem is in ffmpeg, I'll see if I can get vorbis working using libvorbis, so we have at least one lossy encoder. And we can enable MP2 in ffmpeg too, if that one works.

[ttcoder]

That touches on something I'm curious about: seems ffmpeg_x86 is installed "out of the box" on gcc2h, but it does not contain the command line front-end, only the libraries:

~/Desktop> pkgman search ffmpeg
(..)
S       ffmpeg_x86        Audio and video recording, conversion, and streaming library                    
~/Desktop> pkgman install ffmpeg_x86
(..)
Nothing to do.
~/Desktop> query -a ffmpeg_x86
/boot/system/data/ffmpeg_x86
~/Desktop> query -a ffmpeg-x86
~/Desktop> 

Re. enabling mp2 support easily, sounds like a great plan B I could fall back to if needed!

[pulkomandy]

Right, I don't know why the ffmpeg package was setup like this. So at the moment the only way to test the gcc4 ffmpeg is to try a gcc4 version of Haiku. I will see if I can reverse the situation for the gcc2 installs so only the gcc4 version of the binaries is made available. At least we would have something that works.

[waddlesplash]

@pulkomandy Yeah, that sounds like a good idea. And yes it's totally possible to do.

Also, there is a FFmpeg 2.3 recipe in HaikuPorts that I made a few months ago which works on GCC4. I tried ffplay (SDL-based audio/video player that's included by default) and it worked for a few test videos that I had around anyway, so that's a good sign.

But yes, @pulkomandy is right in that we'd have to fork the Media Kit plugin into 2 parts: a GCC2 version for FFmpeg 0.10, and a GCC4 version for 2.3 and beyond as the API is very different (even 0.11 is source/binary incompatible with 0.10).

At any rate, it's certainly possible for @ttcoder to ship a 2.3 ffmpeg binary and library set with TuneTracker and use it to do media conversion internally. It's also possible for us to provide the {{ff*}} commands in the GCC4 package rather than the GCC2 one as well.

[pulkomandy]

I tested this on gcc4 (64bit). Converting with the commands you used above, AC3 works, OGG results in ogg/flac just like MediaConvert does. MP2 also works. The others (vorbis, alac, aac) apparently needs more parameters.

This works fine on both gcc2 and gcc4:

ffmpeg -i test.mp3 -codec:a libvorbis -qcale:a 5 test.ogg

So the ffmpeg library is not at fault, and there is something wrong with our media plugins and/or mediaconvert. This command line makes ffmpeg use libvorbis for audio encoding. There is also a "native" vorbis encoder, which also doesn't crash but has bad quality:

ffmpeg -i test.mp3 -codec:a vorbis -strict experimental -qcale:a 5 test.ogg

[ttcoder]

@waddlesplash With some luck you won't have to change anything in packaging, and we won't have to investigate gcc4 :-b read on.. (BTW before downloading a gcc4h nightly I looked ​here and ​here but was surprised to see the same ffmpeg package listings as I expected the x86 listing would be different from the x86_gcc2 one, tho the list is not easy to make out for a neophyte so leaving that aside for now :-)

@pulkomandy Thanks for going star wars on me and unleashing the power of that fully-operational (ffmpeg) battle station on my ass! I thought it didn't work... And there I am, with an evening and morning spent doing double-blind tests and the such and amazed at how good ffmpeg sounds *g*. Tried using -qscale and -q and I end up not being able to hear the different between the original .wav and the encoded ogg/vorbis even going as low as -q 0 (which encodes to ca. 64 kbps according to MediaPlayer > File Info menu). This laptop has cheapo speakers and I'm not an audiophile but still, I know I've never seen that kind of performance in years of using lame... That one requires 128 kbps or more..

So still todo..

• in my testing the last 6 seconds of original audio are indeed missing in the encoded vorbis audio, as you had originally mentionned.

And..

• have to look for what's breaking ffmpeg libs in 'libvorbis' mode when used by MediaConverter, or used in TT software (TimeTracker, TunePrepper ..etc);

MediaConverter outputs this to stderr/out: (EDIT: on my 47991 partition which still enables vorbis)

[libvorbis @ 0x187b84d0] libvorbis: buffer overflow.
Error writing audio frames: General system error

by opposed to using ffmpeg command-line, which works freakin' awesome:

ffmpeg -i test.wav -acodec libvorbis -q 0 test.ogg

(Not sure if that matters, but DiskProbe and /bin/cmp show that the 4KB (aborted) output from MediaConverter has some difference with the corresponding chunk in command-line ffmpeg's (correct) output: they both start off with '0ggS', 0x200 ..etc but there are some difference starting at offset 0xe. And starting at offset 0x65 there's even a "shift": the correct ogg file contains the string "vorbis" @ 0x67 whereas the incorrect one has it @ 0x68).

[pulkomandy]

I had a look at the code for ffmpeg 0.11 and noticed the fixed size buffer libvorbis complains about was removed. So I'm trying to write a recipe for ffmpeg 0.11.5 and I'll see if I can get the media kit working with that. I have some gcc2 compatibility issues but I should be able to sort it out. I hope there were enough changes in ffmpeg so we don't hit #8856 again.

[pulkomandy]

Oh and about the repositories layout: the "packages" folder is the same for all repositories, and contains all packages (new and old). This saves a considerable amount of disk space on the repo server since we keep old versions of the repository online.

The "repo" file defines what's available in each version of haiku, but is not human readable.

The packages ending with _x86_gcc2 are for gcc2. The packages ending with _x86 are for gcc4. So for a gcc2 install you will be using: ffmpeg-0.10.14-2-x86_gcc2 and ffmpeg_x86-0.10.14-2-x86_gcc2

And for a gcc4 install: ffmpeg-0.10.14-2-x86.hpkg and ffmpeg_x86_gcc2-0.10.14-2-x86.hpkg

The "x86" architecture means gcc4 compiler. This is a bit confusing at first but it allows to keep a simpler naming scheme in the long term.

[ttcoder]

Another odd thing -- ffmpeg-0.11.5-1-x86_gcc2.hpkg is already listed as done ;-) Didn't open it to see what build is inside though

[pulkomandy]

Yes, as I said I'm working on it. That package is broken, and not listed in the public repositories yet. I will upload a working one when I'm done :)

[pulkomandy]

So, I got ffmpeg 0.11.5 to compile and run after a number of hacks. There are some good news: it seems it doesn't break video and audio decoding. So we could actually update to it (#8856 somehow isn't a problem anymore). But there are also some bad news: the vorbis encoding still fails.

However, the error is more obvious this time. What happens is the libvorbis encoder in ffmpeg only can accept floating-point format as an input. I guess most ffmpeg encoders are designed the same. But our media plug-in doesn't propagate this information, and MediaConvert tries to feed integers to the encoder instead. ffmpeg added an explicit check for this in version 0.11, so they error cleanly, but the error isn't handled properly by our plugin and in the end MediaConvert crashes.

[vidrep]

When using the "pkgman full-sync" command, an error message is generated which refers to missing libraries (see attached screenshot).

[vidrep]

A question, if I may. Is there any reason why we cannot go directly to the latest ffmpeg revision in the gcc4 (32 and 64 bit) builds?

[pulkomandy]

Hi, I noticed this issue and am currently preparing release 1.4.6 of haikuwebkit to fix it. I don't know if the build will be done before I leave for BeGeistert, however.

There is a recipe for the latest version of ffmpeg which works with gcc4, but gcc2 will need to stay at 0.10 for now. I did more testing with 0.11 and there are still problems with h263 and h264 decoding. I tried disabling various things (MMX, SSE, use of asm code, etc) but it still crashes. Later versions are even harder to get to compile with gcc2.

There is no problem with updating the gcc4 version of ffmpeg, but the media add-on must be made to work with either version (or maybe we need an ffmpeg_legacy one for gcc2 and an ffmpeg one for gcc4). This allows the gcc4 world to move on but doesn't solve our issue with legacy apps.