MediaPlayer doesn't use overlay altough it is available

[nutela]

grep -i overlay /var/log/syslog KERN: Radeon - ALLOCATE_OVERLAY_BUFFER: success: mem_handle=4, offset=502000, CPU-address=94502000, phys-address=d8502000

MediaPlayer std error:
OpenDMLParser::Parse: JUNK chunk ignored, size: 376 bytes
FormatProposal() error
VideoConsumer::CreateBuffers() - get_area_info(): Operation on invalid team
VideoConsumer::Connected - COULDN'T CREATE BUFFERS
BMediaRoster::Connect: aborting after BBufferConsumer::Connected, status = 0xffffffff
VideoProducer::Connect() - consumer error: General system error
BMediaRoster::Connect: aborted
FormatProposal() error

[euan]

I'd question whether the Radeon overlay is working. Would be good to know what overlay format is being requested.

[stippi]

Interesting. The get_area_info() call is made regardless of overlays. It is needed to map media kit BBuffers to the memory of BBitmaps:

// figure out the bitmap creation flags
uint32 bitmapFlags = 0;
if (fTryOverlay) {
	// try to use hardware overlay
	bitmapFlags |= B_BITMAP_WILL_OVERLAY;
	if (i == 0)
		bitmapFlags |= B_BITMAP_RESERVE_OVERLAY_CHANNEL;
} else
	bitmapFlags = B_BITMAP_IS_LOCKED;
	
fBitmap[i] = new BBitmap(bounds, bitmapFlags, colorSpace);
status = fBitmap[i]->InitCheck();
if (status >= B_OK) {
	buffer_clone_info info;
	uint8* bits = (uint8*)fBitmap[i]->Bits();
	info.area = area_for(bits);
	area_info bitmapAreaInfo;
	status = get_area_info(info.area, &bitmapAreaInfo);
	if (status != B_OK) {
		fprintf(stderr, "VideoConsumer::CreateBuffers() - "
			"get_area_info(): %s\n", strerror(status));
		return status;
	}
	info.offset = bits - (uint8*)bitmapAreaInfo.address;
	info.size = (size_t)fBitmap[i]->BitsLength();
	info.flags = 0;
	info.buffer = 0;
		// the media buffer id
	BBuffer *buffer = NULL;
	if ((status = fBuffers->AddBuffer(info, &buffer)) != B_OK) {
		ERROR("VideoConsumer::CreateBuffers - ERROR ADDING BUFFER TO GROUP\n");
		return status;
	} else
		PROGRESS("VideoConsumer::CreateBuffers - SUCCESSFUL ADD BUFFER TO GROUP\n");
}

If you can watch the video without overlay, it means the exact same code path works for non-overlay bitmaps, which is a bit strange. I guess the team could indeed be different. For normal bitmaps, the app_server would have allocated the area/memory, while for overlay bitmaps, it would have actually been the graphics driver/kernel. Maybe some privilege thing is preventing this from working?