MediaPlayer Streaming issues

[thaflo]

Reproducable on my machine. The stream unterrupts after a while (could be hours). It fills also the RAM.

M~/Desktop> MediaPlayer 
open playlist item: http://stream-dc1.radioparadise.com:80/aac-128
Compiler did not align stack variables. Libavcodec has been miscompiled
and may be very slow or crash. This is not a bug in libavcodec,
but in the compiler. You may try recompiling using gcc >= 4.2.
Do not report crashes to FFmpeg developers.
[aac @ 0x187fbc78] max_analyze_duration 5000000 reached at 5015510
[aac @ 0x187fbc78] Estimating duration from bitrate, this may be inaccurate
[aac @ 0x187fb6b0] max_analyze_duration 5000000 reached at 5015510
[aac @ 0x187fb6b0] Estimating duration from bitrate, this may be inaccurate
MediaFileTrackSupplier: warning! track index 0 has no duration
_InitFromTrack(): keyframes: 1, frame count: 0
running without video node
[aac @ 0x187dc170] Input buffer exhausted before END element found
########### audio decode error, fTempPacket.size 0, fChunkBuffer data offset 0
BMediaTrack::ReadFrames: decoder returned error 0x80004007 (Last buffer)
[aac @ 0x187fbc78] max_analyze_duration 5000000 reached at 5015510
[aac @ 0x187fbc78] Estimating duration from bitrate, this may be inaccurate
[aac @ 0x187dc170] Input buffer exhausted before END element found
########### audio decode error, fTempPacket.size 0, fChunkBuffer data offset 0
BMediaTrack::ReadFrames: decoder returned error 0x80004007 (Last buffer)
[aac @ 0x187dc170] Input buffer exhausted before END element found
########### audio decode error, fTempPacket.size 0, fChunkBuffer data offset 0
BMediaTrack::ReadFrames: decoder returned error 0x80004007 (Last buffer)
[aac @ 0x187dc170] Input buffer exhausted before END element found
BMediaTrack::ReadFrames: decoder returned error 0x80000000 (Out of memory)
[aac @ 0x187dc170] Input buffer exhausted before END element found
BMediaTrack::ReadFrames: decoder returned error 0x80000000 (Out of memory)
[aac @ 0x187dc170] Input buffer exhausted before END element found
BMediaTrack::ReadFrames: decoder returned error 0x80000000 (Out of memory)
[aac @ 0x187dc170] Input buffer exhausted before END element found
BMediaTrack::ReadFrames: decoder returned error 0x80000000 (Out of memory)
[aac @ 0x187dc170] Input buffer exhausted before END element found
BMediaTrack::ReadFrames: decoder returned error 0x80000000 (Out of memory)

[smallstepforman]

On h51659 (x86_64), MediaPlayer is unable to play a Stream URL (​http://naxi128.streaming.rs:9150). Streams fine on StreamRadio. The error reported via the GUI is:

"Error:Out of memory".

On the command line, there are infinite instances of:

"AudioProducer::_FillNextBuffer() - supplier error -> silence"

[leavengood]

Probably related to, or is the same issue as #14047. Will leave both open for now.

[leavengood]

I have added a feature to our debug memory allocator to print allocations periodically. I have run MediaPlayer against the 1980s music stream at ​http://23.92.61.218:9002/ while using this debug feature, printing out the number of allocations and the allocated bytes every 5 seconds. I ran it for quite a while and got the attached output. I have seen similar behavior from the command-line media_client against the same stream.

The summary of this data is that memory starts off at 20 MB or so, then drops to 18, fairly quickly increases to 26, then drops to 16 and then quite slowly increases again to 26 before again dropping to 16 MB. So I feel like something has fixed this issue since it was logged.

Of course in my opinion their should not be all this allocation constantly, but the fact that it builds up, then drops, shows that it isn't leaking forever.

I also just tried the naxi128 stream from smallstepforman and it is also working now (yay Russian music!)

I can try other streams but I feel like we can probably close this issue.

If the OP is still around, could you try testing again with the latest nightly?

[leavengood]

Media Player Allocation record while playing an HTTP stream

[leavengood]

So I am doing a test now with MediaPlayer with the normal allocator and I do not see the same behavior. The memory just seems to keep growing. So this needs more investigation, but the behavior with the debug heap is very interesting as a counterpoint.

[mmlr]

For leak analysis there is also ​the leak analyzer which will show you traces of remaining allocations on program exit based on the leak information of the guarded heap.

It is possible that this is a fragmentation issue, where small allocations remain in such a pattern that some larger ones will not fit in between, leading to more and more heap being allocated over time. The debug heaps manage memory diffrently, so they may not be affected as much by this.

[pulkomandy]

This could be related to the way we handle buffering for these streams. The HTTP request threads downloads the data and fills in a memory buffer (a variant of BMallocIO, IIRC). The data is then read on the other side by the media kit through BMediaIO.

However, it is not clear when we are allowed to forget some previously downloaded data. For a file with a known size/duration, we can keep it all in RAM as long as the BMediaFile is in use (or if the file is too large we could switch to on-disk buffering).

However, for a streaming source, the filesize is essentially infinite. There are flags in BMediaIO but they only allow to guess at this (the source is marked non-seekable, IIRC). There are flags to ffmpeg to tell it about this but I don't know if we properly set them. Only after we have done this, we can know that the ffmpeg plugin will be keeping its own internal buffering of the data, allowing us to make BMediaIO behave as "erase-on-read" in this case: whatever has been read cannot be accessed anymore.

Then, we can somehow make BMediaIO use a fixed size circular buffer for storage, locking the http thread when it downloads faster than the media consumer can use the data, and maybe increase the buffer when this happens, but keeping some sane maximum limit (maybe max 512K of buffer, which should be a dozen seconds for a compressed MP3 file or so?)

[leavengood]

waddlesplash had pointed me to the leak analyzer and that was the first thing I tried, but the guarded heap does not see any leaks in media_client and the only leak it sees in MediaPlayer is related to something minor in the Application Kit.

So whatever is collecting all this memory is properly freed on program exit.

I can definitely look at the streaming code to not have an infinite stream but this also seems to happen in MediaPlayer just going through a playlist consisting of the same short mp3 file, though I think the memory increase is not nearly as fast as with streaming.

I have not tried running MediaPlayer with the playlist using the new periodic dumping I added to the debug heap. But I will so I can see if it does the same build up of memory and then dropping down again.

[thaflo]

Hi leavengood, I'll test it again! Thanks for the work on it!

[thaflo]

Before the start of stream listening

[thaflo]

After listening to the stream for about 90 minutes

[thaflo]

Hi leavengood,

I attached two screenshots. I streamed ​http://stream-uk1.radioparadise.com/aac-320 I did absolutely nothing else then listening between the first and the last minute.

After closing MediaPlayer the RAM usage was about 570 MB.

Greets, Florian

Replying to leavengood:

I have added a feature to our debug memory allocator to print allocations periodically. I have run MediaPlayer against the 1980s music stream at ​http://23.92.61.218:9002/ while using this debug feature, printing out the number of allocations and the allocated bytes every 5 seconds. I ran it for quite a while and got the attached output. I have seen similar behavior from the command-line media_client against the same stream.

The summary of this data is that memory starts off at 20 MB or so, then drops to 18, fairly quickly increases to 26, then drops to 16 and then quite slowly increases again to 26 before again dropping to 16 MB. So I feel like something has fixed this issue since it was logged.

Of course in my opinion their should not be all this allocation constantly, but the fact that it builds up, then drops, shows that it isn't leaking forever.

I also just tried the naxi128 stream from smallstepforman and it is also working now (yay Russian music!)

I can try other streams but I feel like we can probably close this issue.

If the OP is still around, could you try testing again with the latest nightly?

[leavengood]

OK, thank you for testing again. We have determined that we get different behavior with the "debug heap" than for the normal one and I have also seen the "leaking" you noticed. So I continue to try to figure this out...

[Coldfirex]

Could the milestone on this please be updated to R1?