MediaPlayer - make playlist more resilient to errors or corrupt files in the same directory

[vidrep]

Attempted to convert an .avi video file to .mpg Screenshot1 shows source file playing in MediaPlayer with info. Screenshot2 shows source file loaded into MediaConverter with settings Screenshot3 shows both output and source files as unplayable (mediaPlayer crash) Debug report 127682 (output) Debug report 127627 (source)

[anevilyak]

(In #11499) Double submission, #11500 contains more detail/attachments.

[vidrep]

If MediaConverter is not converting the files correctly for whatever reason, that is not a big problem. However, when it renders the SOURCE files unplayable, that is unacceptable.

[pulkomandy]

Is the source file actually modified? Does the md5sum changes? Or is it just that MediaConverter (or the MediaPlayer replay of the output) is confusing the media server or the ffmpeg plug-in and preventing MediaPlayer to replay anything?

[vidrep]

I tried again tonight with the same video. As shown in screenshot2, if RAW audio is selected in the output selection, MediaConverter will render the source file and output unplayable. If FLAC is selected, MediaConverter creates a very small output file, which is also unplayable. However, the source file remains untouched. If FLAC and Theora are selected it will convert properly. I' transferred the "damaged" source file to Windows, and it played normally. So, apparently MediaConverter is somehow altering the source file in such a way that MediaPlayer cannot recognize it as a valid media file after it is converted.

[pulkomandy]

Can you answer my questions please? See if the md5sum of the source file changes or not.

[vidrep]

I used the command "md5sum test1.avi > test1.txt" to create and output a md5 hash for the source file to be converted. After converting to .mpeg in MediaConverter, I checked to see if the md5sum of the now unplayable source file had changed. It did not change. The output file from the conversion is very small in size (3.85 MiB) compared to the source (86.74 MiB).

[anevilyak]

The fact that the md5 doesn't change means that the source file isn't in fact bring modified. This would imply that what's actually happening is that something in either one of the add-ons, or in the add-on server is breaking as a result of the conversion request. Can you confirm that the source file is playable again after a reboot?

[vidrep]

After reboot it still does not play - MediaPlayer crashes. However, if I copy the file onto a mounted NTFS volume (such as a USB stick) and try to play it with MediaPlayer, it plays fine.

[ttcoder]

Maybe that hints at a modified attribute (rather than core data) change then, there's a couple that are written when opening a file in MediaPlayer, I suppose it's not impossible some attributes are written to the source file when opened in MediaConverter too.. (what happens if you copy the file back from NTFS to a BFS partition? it should be stripped of its attributes then).

Can you do a before-and-after listattr file-name-here (similar to what you did with md5).. And since that only dumps the list but not the attribute contents, also do catattr Media:Length file-name-goes-here, before-and-after, and really, other attributes would be good to have too.

[vidrep]

I tried your suggestion. I copied the file back onto my Desktop from the NTFS fomatted USB stick and now MediaPlayer plays the file again. However, if the source file and converted output file are in the same directory together (i.e. test1.avi --> test1.mpg), neither will play - MediaPlayer will crash. If I move the converted output file out to another directory, the source file plays OK again.

Source After Conversion

~> listattr test1.avi
File: test1.avi
        Type       Size  Name                                
----------------------------------------------------------
    Raw Data        20  "_trk/pinfo_le"
 MIME String        16  "BEOS:TYPE"
      Int-64         8  "Media:Length"
        Text         7  "Video:Bitrate"

51 bytes total in attributes

~/Desktop/Home> catattr Media:Length test1.avi
test1.avi : int64 : 45832875

Copy From NTFS Volume

~/Desktop> listattr test1.avi
File: test1.avi
        Type       Size  Name                                
----------------------------------------------------------
    Raw Data        20  "_trk/pinfo_le"
 MIME String        16  "BEOS:TYPE"
      Int-64         8  "Media:Length"
        Text         7  "Video:Bitrate"

51 bytes total in attributes.

~/Desktop> catattr Media:Length test1.avi
test1.avi : int64 : 45832875

Converted Output File

~> listattr test1.mpg
File: test1.mpg
        Type       Size  Name                                
----------------------------------------------------------
 MIME String        11  "BEOS:TYPE"
        Text        21  "_trk/original_path"

32 bytes total in attributes.

~> catattr Media:Length test1.mpg
catattr: "test1.mpg", attribute "Media:Length": No such file or directory

[ttcoder]

Replying to vidrep:

However, if the source file and converted output file are in the same directory together (i.e. test1.avi --> test1.mpg), neither will play - MediaPlayer will crash. If I move the converted output file out to another directory, the source file plays OK again.

Eureka!.. -- I think I know exactly what you are talking of, though your symptoms are exactly in reverse of mine, so it took me a while to figure it out.

Details.. Until a few weeks ago, media conversion was permitted on a long list of formats, including those formats that did not work, like ogg/vorbis ..etc. Trying to convert media (audio, in my case) to those formats would produce tiny (header-only) files of e.g. 58 bytes. And the weird thing was.. double-clicking those files to open them in MediaPlayer would play vibrant, high quality, long duration audio.. Obviously a physical (or mathematical) impossibility on its face *grin*. When I first saw that a year or so ago I freaked out, then quickly realized I was not crazy, it was simply that MediaConverter was playing, not the .ogg file I had double-clicked, but the original .wav (source) file, i.e. the file that came next in alphabetical order.

In vidrep's use-case, the conversion is "test.avi" to "test.mpg", i.e. it's the converted file that comes next in alphabetical order, so clicking the source file could attempt to play the converted file, and crash since it's just an almost-empty stub file.

@pulkomandy -- the bug I was seeing then was really the weirdest of bugs, and if that's what vidrep is seeing too, well.. might as well fix it ASAP! So.. steps to reproduce:

• make a clean/empty folder
• create/copy a normally playing test.wav file in there
• create a tiny file with a similar name, that would come earlier in alphabetical order (e.g. test.ogg ?).. I'm not sure if the bug is contents dependant, or MIME-type dependant. Try to make the test case as close as possible to what MediaConverter creates when broken (or revert to an earlier version of Haiku/ffmpeg that was broken ;-)
• double click the 50-bytes file.. and be amazed at how well it plays for such a small size *g*

If MediaPlayer actually playes the multi-megabytes .wav file instead, there goes your reproducible case for me (and likely for vidrep).

[vidrep]

I tried converting the file using different combinations of audio and video encoders in MPEG format, and only the combination described in the ticket produces the non-playable source file symptom. Some other combinations also produced small header files but didn't affect the source file. So, it appears that the alphabetical order theory may be incorrect. Over the next couple of days I'll try to document the results of different encoder combinations in a spreadsheet, and attach it to the ticket. The developers may have to consider only including working combinations in the Beta until all the bugs are worked out.

[richienyhus]

Replying to vidrep:

The developers may have to consider only including working combinations in the Beta until all the bugs are worked out.

Since this bug results in data loss, shouldn't this be a beta blocker until it is fixed or until the offending components are removed from images?

[pulkomandy]

As mentionned in the comments above, the MD5 sum of the source file is not changed, which means there is no data corruption here. So, no need for a beta blocker (MediaConvert was even more broken in the previous releases).

[richienyhus]

Replying to pulkomandy:

As mentionned in the comments above, the MD5 sum of the source file is not changed, which means there is no data corruption here. So, no need for a beta blocker (MediaConvert was even more broken in the previous releases).

OK then, I must have skipped over that part of the conversation.

[ttcoder]

Yup.. I would put emphasis on the following from comment:10 , quotted below.

In other words, the crash/misbehaving occurs only if 1) the file is not stripped of its attributes (by a roundtrip to e.g. ntfs) and 2) it lives beside a file with a similar name but different extension. So it's contextual, very different animal than if the file contents itself was changed and could no longer be played any more, i.e. corrupted.. Weird stuff though.

I tried your suggestion. I copied the file back onto my Desktop from the NTFS fomatted USB stick and now MediaPlayer plays the file again. However, if the source file and converted output file are in the same directory together (i.e. test1.avi --> test1.mpg), neither will play - MediaPlayer will crash. If I move the converted output file out to another directory, the source file plays OK again.

[vidrep]

We know that the converted file shouldn't reside in the same folder as the source file, but the end user who may be trying Haiku for the first time isn't going to know this. MediaConverter should create its own default directory for converted files. Many Windows programs do exactly this as well. Perhaps for similar reasons?

[pulkomandy]

What about fixing the actual problem, so this setup just works? Instead of adding artificial limitations to hide the bug (which could trigger in completely different situations?)

[pulkomandy]

Ok, after looking at the code in MediaPlayer for subtitles, I found what is happening.

When you load a file in MediaPlayer, the playlist management will look for other files in the directory, with the same name and different extensions. It will load them and offer them as:

• Subtitles
• Alternative audio and video tracks

The use case for this is, for example, I have a movie with English sound, and an MP3 file with the French soundtrack. Both are loaded and I can switch from MediaPlayer "audio" menu.

If you use MediaConverter and put the output files in the same directory as the source, then MediaPlayer will load both. And if one of them fails or has no audio track, it will just use the other.

The good news is that your source files are not unplayable, they are just disturbed by the presence of converted/corrupt files in the same directory. And the solution is simply to move these files elsewhere.

So it turns out MediaConverter isn't at fault here, however probably MediaPlayer should be made more resilient to errors in this case, and maybe the UI tweaked to better show that this is happening.