Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: f114890c78b5bb1de5d09d133a3b702618d02a06
https://github.com/WebKit/WebKit/commit/f114890c78b5bb1de5d09d133a3b702618d02a06
Author: Andy Estes <aes...@apple.com>
Date: 2024-04-26 (Fri, 26 Apr 2024)
Changed paths:
M LayoutTests/platform/mac/TestExpectations
M Source/WebCore/html/HTMLVideoElement.cpp
M Source/WebCore/platform/graphics/MediaPlayer.cpp
M Source/WebCore/platform/graphics/MediaPlayer.h
M Source/WebCore/platform/graphics/MediaPlayerPrivate.h
M Source/WebCore/platform/graphics/SourceBufferPrivate.cpp
M Source/WebCore/platform/graphics/SourceBufferPrivate.h
M
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h
M
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm
M
Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h
M
Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm
M
Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h
M
Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm
M Source/WebCore/platform/graphics/cocoa/WebSampleBufferVideoRendering.h
M Source/WebCore/platform/ios/VideoPresentationInterfaceAVKit.h
M Source/WebCore/platform/ios/VideoPresentationInterfaceAVKit.mm
M Source/WebCore/platform/ios/VideoPresentationInterfaceIOS.h
M Source/WebCore/platform/ios/VideoPresentationInterfaceIOS.mm
M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp
M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h
M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in
M Source/WebKit/Platform/ios/VideoPresentationInterfaceLMK.h
M Source/WebKit/Platform/ios/VideoPresentationInterfaceLMK.mm
M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp
M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h
Log Message:
-----------
[visionOS] video goes blank and audio pauses when transitioning to
LinearMediaKit fullscreen
https://bugs.webkit.org/show_bug.cgi?id=273171
rdar://125647233
Reviewed by Jer Noble.
When transitioning to LinearMediaKit fullscreen, two issues cause the video to
pause and show a
black frame in the WKWebView:
1. VideoPresentationInterfaceIOS moves the video view from the web view to a
new WebAVPlayerView,
even though this view is not used by LMPlayableViewController.
2. When MediaPlayerPrivateMediaSourceAVFObjC switches from an
AVSampleBufferDisplayLayer to an
AVSampleBufferVideoRenderer, it flushes the display layer and sets the
synchronizer's rate to 0
until the video render has an available frame.
Resolved (1) by keeping the video view in the WKWebView's hierarchy when
VideoPresentationInterfaceLMK is in use. This is OK since
LMPLayableViewController will render to an
entity rather than a layer.
Resolved (2) by teaching MediaPlayerPrivateMediaSourceAVFObjC to stage the
transition from
AVSampleBufferDisplayLayer to AVSampleBufferVideoRenderer (and vice versa).
When a video receiver
endpoint is received an AVSampleBufferVideoRenderer is created, but the existing
AVSampleBufferDisplayLayer is kept alive. It isn't flushed, so it continues to
render samples that
have been previously enqueued. Simultaneously, samples for the current time are
enqueued to the
AVSampleBufferVideoRenderer, and only once the video render has an available
frame the display layer
is destroyed. The synchronizer remains playing during this transition. A
similar staged transition
occurs when switching back from a video renderer to a display layer.
* Source/WebCore/html/HTMLVideoElement.cpp:
(WebCore::HTMLVideoElement::didEnterFullscreenOrPictureInPicture):
(WebCore::HTMLVideoElement::didExitFullscreenOrPictureInPicture):
* Source/WebCore/platform/graphics/MediaPlayer.cpp:
(WebCore::MediaPlayer::setInFullscreenOrPictureInPicture):
(WebCore::MediaPlayer::isInFullscreenOrPictureInPicture const):
* Source/WebCore/platform/graphics/MediaPlayer.h:
* Source/WebCore/platform/graphics/MediaPlayerPrivate.h:
(WebCore::MediaPlayerPrivateInterface::isInFullscreenOrPictureInPictureChanged):
* Source/WebCore/platform/graphics/SourceBufferPrivate.cpp:
(WebCore::SourceBufferPrivate::reenqueSamples):
(WebCore::SourceBufferPrivate::reenqueueMediaForTime):
* Source/WebCore/platform/graphics/SourceBufferPrivate.h:
*
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h:
*
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::load):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::updateLastPixelBuffer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::shouldEnsureLayerOrVideoRenderer
const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setPresentationSize):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setVideoLayerSizeFenced):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::updateDisplayLayerAndDecompressionSession):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::videoPlaybackQualityMetrics):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureLayer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyLayer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureVideoRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyVideoRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureDecompressionSession):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyDecompressionSession):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureLayerOrVideoRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyLayerOrVideoRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::configureLayerOrVideoRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::acceleratedVideoMode const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::playerContentBoxRectChanged):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::layerOrVideoRenderer const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setVideoReceiverEndpoint):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::isInFullscreenOrPictureInPictureChanged):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::shouldEnsureLayer const):
Deleted.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::shouldEnsureVideoRenderer
const): Deleted.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::sampleBufferVideoRenderer
const): Deleted.
*
Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h:
*
Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm:
(WebCore::MediaSourcePrivateAVFObjC::stageVideoRenderer):
(WebCore::MediaSourcePrivateAVFObjC::setSourceBufferWithSelectedVideo):
*
Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
*
Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::SourceBufferPrivateAVFObjC::videoRendererReadyForDisplayChanged):
(WebCore::SourceBufferPrivateAVFObjC::enqueueSampleBuffer):
(WebCore::SourceBufferPrivateAVFObjC::configureVideoRenderer):
(WebCore::SourceBufferPrivateAVFObjC::invalidateVideoRenderer):
(WebCore::SourceBufferPrivateAVFObjC::setVideoRenderer):
(WebCore::SourceBufferPrivateAVFObjC::stageVideoRenderer):
* Source/WebCore/platform/graphics/cocoa/WebSampleBufferVideoRendering.h:
* Source/WebCore/platform/ios/VideoPresentationInterfaceAVKit.h:
* Source/WebCore/platform/ios/VideoPresentationInterfaceAVKit.mm:
(WebCore::VideoPresentationInterfaceAVKit::willRenderToLayer const):
* Source/WebCore/platform/ios/VideoPresentationInterfaceIOS.h:
* Source/WebCore/platform/ios/VideoPresentationInterfaceIOS.mm:
(WebCore::VideoPresentationInterfaceIOS::doSetup):
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp:
(WebKit::RemoteMediaPlayerProxy::isInFullscreenOrPictureInPictureChanged):
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
* Source/WebKit/Platform/ios/VideoPresentationInterfaceLMK.h:
* Source/WebKit/Platform/ios/VideoPresentationInterfaceLMK.mm:
(WebKit::VideoPresentationInterfaceLMK::finalizeSetup):
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
(WebKit::MediaPlayerPrivateRemote::isInFullscreenOrPictureInPictureChanged):
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
Canonical link: https://commits.webkit.org/278032@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes
