Changes between Version 1 and Version 2 of MediaKitImprovements

Mar 19, 2019, 8:05:43 PM (17 months ago)



  • MediaKitImprovements

    v1 v2  
    11This is a set of notes about Media Kit current limitations and improvements, attempting to extract the core ideas that led Barrett to attempt to write a Media2 and a Codec Kit. The goal is to see how much of this can be implemented while preserving the current APIs, and what needs to be moved to R2 or a new kit.
    3 Design
    4 ======
     3= Design =
    65Media nodes are a pain to work with. There is a lot of boilerplate code to write to get one up and running. An easier API could maybe be considered, perhaps as a wrapper around the existing infrastructure.
    87The code for realtime streams and for media file encoding/decoding are not clearly separated, while the implementations are in fact largely independant. Without going to the extreme of two separate kits, at least moving the sourcefiles to different subdirectories may clarify this a little. This separation is questionable, for example DirectShow ( implements file reading (BMediaFile), splitting (BMediaTrack) and decoding as separate "filters".
    10 Non audio/video use
    11 ================
     9= Non audio/video use =
    1311The two main use cases here are subtitles (not very complicated, just a new kind of stream that goes in the same direction as audio and video data), and DVD/BlueRay menus. Blueray is quite complicated (essentially, BlueRays can run arbitrary code in a Java virtual machine), so it's questionable if the Media Kit is the appropriate place to support this. DVD might be simple enough to be workable, but requires getting mouse and keyboard events to run the menu, which goes in the reverse direction from the usual media stream.
    1513Another possible one is midi events, currently covered in a separate kit, but which are also a realtime stream of arbitrary data.
    17 Streaming
    18 ========
     15= Streaming =
    2017BMediaFile (as the name implies) was not really designed with streaming in mind. It has been implemented, still, but there are limitations. In particular, currently sniffing the file is done synchronously in the constructor and blocks until data has been downloaded, and there is no way to manage the download buffer to have proper seeking properties. It looks like neither of these need an API change, however.
    22 Input/output formats
    23 =================
     19= Input/output formats =
    2521The current API was designed in the days of AVI and QuickTime and revolves around 4CCs and other limited ways to identify content types. MIME type might be used instead, but they may not be appropriate for stream encoding format (they are more designed for file format). In any case, a review of how we specify the types is probably needed.
    27 Non-FFMPEG encoding/decoding
    28 =========================
     23= Non-FFMPEG encoding/decoding =
    3025It would be great to make sure we are not restricted to ffmpeg. This may include unusual usecases (for example, an 8 bit computer emulator wanting to encode or decode tape data from a tape image, no matter if recorded as wav file, played from computer line in, or loaded from a more compact specific tape image format).