Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 90086fffd11ed100b343d615c03cf446a59734e0
https://github.com/WebKit/WebKit/commit/90086fffd11ed100b343d615c03cf446a59734e0
Author: Jean-Yves Avenard <j...@apple.com>
Date: 2024-02-09 (Fri, 09 Feb 2024)
Changed paths:
M Source/WTF/wtf/Forward.h
M Source/WebCore/Modules/mediasource/ManagedMediaSource.cpp
M Source/WebCore/Modules/mediasource/ManagedMediaSource.h
M Source/WebCore/Modules/mediasource/MediaSource.cpp
M Source/WebCore/Modules/mediasource/MediaSource.h
M Source/WebCore/Modules/mediasource/SourceBuffer.cpp
M Source/WebCore/Modules/mediasource/SourceBuffer.h
M Source/WebCore/platform/graphics/MediaSourcePrivate.cpp
M Source/WebCore/platform/graphics/MediaSourcePrivate.h
M Source/WebCore/platform/graphics/MediaSourcePrivateClient.h
M Source/WebCore/platform/graphics/SourceBufferPrivate.cpp
M Source/WebCore/platform/graphics/SourceBufferPrivate.h
M Source/WebCore/platform/graphics/SourceBufferPrivateClient.h
M Source/WebKit/CMakeLists.txt
M Source/WebKit/DerivedSources-input.xcfilelist
M Source/WebKit/DerivedSources-output.xcfilelist
M Source/WebKit/DerivedSources.make
M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp
M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h
M Source/WebKit/GPUProcess/media/RemoteMediaSourceProxy.cpp
M Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp
M Source/WebKit/WebKit.xcodeproj/project.pbxproj
M Source/WebKit/WebProcess/GPU/GPUProcessConnection.cpp
M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp
M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h
M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.messages.in
M Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.cpp
M Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.h
R Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.messages.in
A
Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemoteMessageReceiver.h
A
Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemoteMessageReceiver.messages.in
M Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerState.h
M Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerState.serialization.in
M Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp
M Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h
R Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.messages.in
A
Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemoteMessageReceiver.h
A
Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemoteMessageReceiver.messages.in
Log Message:
-----------
[MSE] Run MediaSourcePrivateRemote and SourceBufferPrivateRemote in a
dedicated work queue
https://bugs.webkit.org/show_bug.cgi?id=265982
rdar://119296319
Reviewed by Jer Noble.
Make the MediaSourcePrivateRemote and SourceBufferPrivateRemote use a
WorkQueueMessageReceiver
with a dedicated WorkQueue.
The threading model in the GPU process remains unchanged as the current
implementations
and relationship between SourceBufferPrivate and MediaPlayerPrivate rely
heavily on the objects being main-thread only (in particular the extensive use
of
AVSampleBufferDisplayerLayer).
>From an observer point of view in the content process, the various MSE objects
>operations
are now fully parallelised.
The threading flow as such is now:
Content Process:
MediaSource (main thread) -> MediaSourcePrivateRemote (workqueue)
SourceBuffer (main thread) -> SourceBufferPrivateRemote (workqueue)
GPU Process:
RemoteMediaSourceProxy (main thread) -> MediaSourcePrivate (main thread)
RemoteSourceBufferProxy (main thread) -> SourceBufferPrivate (main thread)
The MediaPlayerPrivate/MediaPlayerPrivateRemote runs on the main thread.
Some MediaSourcePrivate methods had to be made thread-safe to more easily
synchronise member attributes.
Covered by existing tests, no change in observable behaviour.
* Source/WTF/wtf/Forward.h:
Add SerialFunctionDispatcher and RefCountedSerialFunctionDispatcher forward
declaration.
* Source/WebCore/Modules/mediasource/ManagedMediaSource.cpp:
(WebCore::ManagedMediaSource::elementDetached):
(WebCore::ManagedMediaSource::setStreaming):
(WebCore::ManagedMediaSource::monitorSourceBuffers):
* Source/WebCore/Modules/mediasource/ManagedMediaSource.h:
* Source/WebCore/Modules/mediasource/MediaSource.cpp:
(WebCore::MediaSource::MediaSource):
(WebCore::MediaSource::waitForTarget):
(WebCore::MediaSource::completeSeek):
(WebCore::MediaSource::seekToTime):
(WebCore::MediaSource::seekable):
(WebCore::MediaSource::setLiveSeekableRange):
(WebCore::MediaSource::clearLiveSeekableRange):
(WebCore::MediaSource::hasBufferedTime):
(WebCore::MediaSource::hasFutureTime):
(WebCore::MediaSource::monitorSourceBuffers):
(WebCore::MediaSource::setDuration):
(WebCore::MediaSource::setDurationInternal):
(WebCore::MediaSource::setReadyState):
(WebCore::MediaSource::endOfStream):
(WebCore::MediaSource::streamEndedWithError):
(WebCore::MediaSource::addSourceBuffer):
(WebCore::MediaSource::detachFromElement):
(WebCore::MediaSource::onReadyStateChange):
(WebCore::MediaSource::activeRanges const):
(WebCore::MediaSource::regenerateActiveSourceBuffers):
(WebCore::MediaSource::notifyElementUpdateMediaState const):
(WebCore::MediaSource::sourceBufferBufferedChanged):
(WebCore::MediaSource::updateBufferedIfNeeded):
(WebCore::MediaSource::failedToCreateRenderer):
(WebCore::MediaSource::sourceBufferReceivedFirstInitializationSegmentChanged):
(WebCore::MediaSource::sourceBufferActiveTrackFlagChanged):
(WebCore::MediaSource::setMediaPlayerReadyState):
(WebCore::MediaSource::memoryPressure):
(WebCore::MediaSource::ensureWeakOnDispatcher const):
* Source/WebCore/Modules/mediasource/MediaSource.h:
(WebCore::MediaSource::elementDetached):
(WebCore::MediaSource::promisedWeakOnDispatcher const):
* Source/WebCore/Modules/mediasource/SourceBuffer.cpp:
(WebCore::m_logIdentifier):
(WebCore::SourceBuffer::~SourceBuffer):
(WebCore::SourceBuffer::rangeRemoval):
(WebCore::SourceBuffer::removedFromMediaSource):
(WebCore::SourceBuffer::seekToTime):
(WebCore::SourceBuffer::appendBufferInternal):
(WebCore::SourceBuffer::sourceBufferPrivateDidReceiveRenderingError):
(WebCore::SourceBuffer::maximumBufferSize const):
(WebCore::SourceBuffer::setActive):
(WebCore::SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment):
(WebCore::SourceBuffer::sourceBufferPrivateDurationChanged):
(WebCore::SourceBuffer::sourceBufferPrivateHighestPresentationTimestampChanged):
(WebCore::SourceBuffer::sourceBufferPrivateDidDropSample):
(WebCore::SourceBuffer::sourceBufferPrivateBufferedChanged):
(WebCore::SourceBuffer::ensureWeakOnDispatcher const):
(WebCore::SourceBuffer::promisedWeakOnDispatcher const):
* Source/WebCore/Modules/mediasource/SourceBuffer.h:
* Source/WebCore/platform/graphics/MediaSourcePrivate.cpp:
(WebCore::MediaSourcePrivate::MediaSourcePrivate):
(WebCore::MediaSourcePrivate::duration const):
(WebCore::MediaSourcePrivate::removeSourceBuffer):
(WebCore::MediaSourcePrivate::sourceBufferPrivateDidChangeActiveState):
(WebCore::MediaSourcePrivate::hasAudio const):
(WebCore::MediaSourcePrivate::hasVideo const):
(WebCore::MediaSourcePrivate::durationChanged):
(WebCore::MediaSourcePrivate::bufferedChanged):
(WebCore::MediaSourcePrivate::buffered const):
(WebCore::MediaSourcePrivate::setCDMSession):
(WebCore::MediaSourcePrivate::ensureOnDispatcher const):
* Source/WebCore/platform/graphics/MediaSourcePrivate.h:
* Source/WebCore/platform/graphics/MediaSourcePrivateClient.h:
* Source/WebCore/platform/graphics/SourceBufferPrivate.cpp:
(WebCore::SourceBufferPrivate::SourceBufferPrivate):
(WebCore::SourceBufferPrivate::removedFromMediaSource):
(WebCore::SourceBufferPrivate::setClient):
(WebCore::SourceBufferPrivate::updateBuffered):
(WebCore::SourceBufferPrivate::seekToTime):
(WebCore::SourceBufferPrivate::removeCodedFrames):
(WebCore::SourceBufferPrivate::updateTrackIds):
(WebCore::SourceBufferPrivate::setAllTrackBuffersNeedRandomAccess):
(WebCore::SourceBufferPrivate::didReceiveInitializationSegment):
(WebCore::SourceBufferPrivate::didUpdateFormatDescriptionForTrackId):
(WebCore::SourceBufferPrivate::append):
(WebCore::SourceBufferPrivate::resetParserState):
(WebCore::SourceBufferPrivate::memoryPressure):
(WebCore::SourceBufferPrivate::ensureOnDispatcher const):
(WebCore::SourceBufferPrivate::detach): Deleted.
* Source/WebCore/platform/graphics/SourceBufferPrivate.h:
* Source/WebCore/platform/graphics/SourceBufferPrivateClient.h:
* Source/WebKit/CMakeLists.txt:
* Source/WebKit/DerivedSources-input.xcfilelist:
* Source/WebKit/DerivedSources-output.xcfilelist:
* Source/WebKit/DerivedSources.make:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp:
(WebKit::RemoteMediaPlayerProxy::commitAllTransactions):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerNetworkStateChanged):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerReadyStateChanged):
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h:
* Source/WebKit/GPUProcess/media/RemoteMediaSourceProxy.cpp:
(WebKit::RemoteMediaSourceProxy::waitForTarget):
(WebKit::RemoteMediaSourceProxy::seekToTime):
(WebKit::RemoteMediaSourceProxy::shutdown):
* Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp:
(WebKit::RemoteSourceBufferProxy::~RemoteSourceBufferProxy):
(WebKit::RemoteSourceBufferProxy::sourceBufferPrivateDidReceiveInitializationSegment):
(WebKit::RemoteSourceBufferProxy::sourceBufferPrivateHighestPresentationTimestampChanged):
(WebKit::RemoteSourceBufferProxy::sourceBufferPrivateDurationChanged):
(WebKit::RemoteSourceBufferProxy::sourceBufferPrivateBufferedChanged):
(WebKit::RemoteSourceBufferProxy::sourceBufferPrivateDidDropSample):
(WebKit::RemoteSourceBufferProxy::sourceBufferPrivateDidReceiveRenderingError):
(WebKit::RemoteSourceBufferProxy::append):
* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebProcess/GPU/GPUProcessConnection.cpp:
(WebKit::GPUProcessConnection::dispatchMessage):
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
(WebKit::MediaPlayerPrivateRemote::setReadyState):
(WebKit::MediaPlayerPrivateRemote::readyStateChanged):
(WebKit::MediaPlayerPrivateRemote::updateCachedState):
(WebKit::MediaPlayerPrivateRemote::addRemoteAudioTrack):
(WebKit::MediaPlayerPrivateRemote::removeRemoteAudioTrack):
(WebKit::MediaPlayerPrivateRemote::remoteAudioTrackConfigurationChanged):
(WebKit::MediaPlayerPrivateRemote::addRemoteTextTrack):
(WebKit::MediaPlayerPrivateRemote::removeRemoteTextTrack):
(WebKit::MediaPlayerPrivateRemote::remoteTextTrackConfigurationChanged):
(WebKit::MediaPlayerPrivateRemote::parseWebVTTFileHeader):
(WebKit::MediaPlayerPrivateRemote::parseWebVTTCueData):
(WebKit::MediaPlayerPrivateRemote::parseWebVTTCueDataStruct):
(WebKit::MediaPlayerPrivateRemote::addDataCue):
(WebKit::MediaPlayerPrivateRemote::addDataCueWithType):
(WebKit::MediaPlayerPrivateRemote::updateDataCue):
(WebKit::MediaPlayerPrivateRemote::removeDataCue):
(WebKit::MediaPlayerPrivateRemote::addGenericCue):
(WebKit::MediaPlayerPrivateRemote::updateGenericCue):
(WebKit::MediaPlayerPrivateRemote::removeGenericCue):
(WebKit::MediaPlayerPrivateRemote::addRemoteVideoTrack):
(WebKit::MediaPlayerPrivateRemote::removeRemoteVideoTrack):
(WebKit::MediaPlayerPrivateRemote::remoteVideoTrackConfigurationChanged):
(WebKit::MediaPlayerPrivateRemote::requestResource):
(WebKit::MediaPlayerPrivateRemote::removeResource):
(WebKit::MediaPlayerPrivateRemote::audioTrackPrivateRemote const):
(WebKit::MediaPlayerPrivateRemote::videoTrackPrivateRemote const):
(WebKit::MediaPlayerPrivateRemote::textTrackPrivateRemote const):
(WebKit::MediaPlayerPrivateRemote::commitAllTransactions):
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.messages.in:
* Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.cpp:
(WebKit::MediaSourcePrivateRemote::queue):
(WebKit::MediaSourcePrivateRemote::ensureOnDispatcherSync const):
(WebKit::MediaSourcePrivateRemote::MediaSourcePrivateRemote):
(WebKit::MediaSourcePrivateRemote::~MediaSourcePrivateRemote):
(WebKit::MediaSourcePrivateRemote::addSourceBuffer):
(WebKit::MediaSourcePrivateRemote::durationChanged):
Directly notify the MediaPlayerPrivateRemote that the duration as changed
rather than go through two round-trips to the GPU process.
(WebKit::MediaSourcePrivateRemote::bufferedChanged):
(WebKit::MediaSourcePrivateRemote::markEndOfStream):
(WebKit::MediaSourcePrivateRemote::unmarkEndOfStream):
(WebKit::MediaSourcePrivateRemote::setMediaPlayerReadyState):
Directly notify the MediaPlayerPrivateRemote that the readyState as changed
rather than go through two round-trips to the GPU process.
(WebKit::MediaSourcePrivateRemote::setTimeFudgeFactor):
(WebKit::MediaSourcePrivateRemote::MessageReceiver::client const):
(WebKit::MediaSourcePrivateRemote::MessageReceiver::proxyWaitForTarget):
(WebKit::MediaSourcePrivateRemote::MessageReceiver::proxySeekToTime):
(WebKit::MediaSourcePrivateRemote::MessageReceiver::mediaSourcePrivateShuttingDown):
(WebKit::MediaSourcePrivateRemote::MessageReceiver::MessageReceiver):
(WebKit::MediaSourcePrivateRemote::proxyWaitForTarget): Deleted.
(WebKit::MediaSourcePrivateRemote::proxySeekToTime): Deleted.
(WebKit::MediaSourcePrivateRemote::mediaSourcePrivateShuttingDown): Deleted.
* Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemoteMessageReceiver.h:
Copied from
Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.messages.in.
The current infrastructure doesn't allow to define the IPC protocol in a
private sub-class, so add an alias.
*
Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemoteMessageReceiver.messages.in:
Copied from
Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.messages.in.
* Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerState.h:
Remove readyState from being state caching structure. As the readyState was
updated alongside other attributes, it was possible
for the readyState to be upgraded with a GPU's stale value before updated to
the new value from the content process. Leading to intermittent
failures, that those changes made more likely.
* Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerState.serialization.in:
* Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp:
(WebKit::SourceBufferPrivateRemote::append):
(WebKit::SourceBufferPrivateRemote::isActive const):
(WebKit::SourceBufferPrivateRemote::setActive):
(WebKit::SourceBufferPrivateRemote::setMediaSourceEnded):
(WebKit::SourceBufferPrivateRemote::queue):
(WebKit::SourceBufferPrivateRemote::ensureOnDispatcherSync):
(WebKit::SourceBufferPrivateRemote::SourceBufferPrivateRemote):
(WebKit::SourceBufferPrivateRemote::~SourceBufferPrivateRemote):
(WebKit::SourceBufferPrivateRemote::append):
(WebKit::SourceBufferPrivateRemote::abort):
(WebKit::SourceBufferPrivateRemote::resetParserState):
(WebKit::SourceBufferPrivateRemote::removedFromMediaSource):
(WebKit::SourceBufferPrivateRemote::canSwitchToType):
(WebKit::SourceBufferPrivateRemote::setMediaSourceEnded):
(WebKit::SourceBufferPrivateRemote::setMode):
(WebKit::SourceBufferPrivateRemote::removeCodedFrames):
(WebKit::SourceBufferPrivateRemote::evictCodedFrames):
(WebKit::SourceBufferPrivateRemote::addTrackBuffer):
(WebKit::SourceBufferPrivateRemote::resetTrackBuffers):
(WebKit::SourceBufferPrivateRemote::clearTrackBuffers):
(WebKit::SourceBufferPrivateRemote::setAllTrackBuffersNeedRandomAccess):
(WebKit::SourceBufferPrivateRemote::setGroupStartTimestamp):
(WebKit::SourceBufferPrivateRemote::setGroupStartTimestampToEndTimestamp):
(WebKit::SourceBufferPrivateRemote::setShouldGenerateTimestamps):
(WebKit::SourceBufferPrivateRemote::reenqueueMediaIfNeeded):
(WebKit::SourceBufferPrivateRemote::resetTimestampOffsetInTrackBuffers):
(WebKit::SourceBufferPrivateRemote::startChangingType):
(WebKit::SourceBufferPrivateRemote::timestampOffset const):
(WebKit::SourceBufferPrivateRemote::setTimestampOffset):
(WebKit::SourceBufferPrivateRemote::setAppendWindowStart):
(WebKit::SourceBufferPrivateRemote::setAppendWindowEnd):
(WebKit::SourceBufferPrivateRemote::computeSeekTime):
(WebKit::SourceBufferPrivateRemote::seekToTime):
(WebKit::SourceBufferPrivateRemote::updateTrackIds):
(WebKit::SourceBufferPrivateRemote::bufferedSamplesForTrackId):
(WebKit::SourceBufferPrivateRemote::enqueuedSamplesForTrackID):
(WebKit::SourceBufferPrivateRemote::MessageReceiver::client const):
(WebKit::SourceBufferPrivateRemote::MessageReceiver::takeOwnershipOfMemory):
(WebKit::SourceBufferPrivateRemote::MessageReceiver::sourceBufferPrivateDidReceiveInitializationSegment):
(WebKit::SourceBufferPrivateRemote::MessageReceiver::sourceBufferPrivateHighestPresentationTimestampChanged):
(WebKit::SourceBufferPrivateRemote::MessageReceiver::sourceBufferPrivateDurationChanged):
(WebKit::SourceBufferPrivateRemote::MessageReceiver::sourceBufferPrivateBufferedChanged):
(WebKit::SourceBufferPrivateRemote::MessageReceiver::sourceBufferPrivateDidDropSample):
(WebKit::SourceBufferPrivateRemote::MessageReceiver::sourceBufferPrivateDidReceiveRenderingError):
(WebKit::SourceBufferPrivateRemote::memoryPressure):
(WebKit::SourceBufferPrivateRemote::minimumUpcomingPresentationTimeForTrackID):
(WebKit::SourceBufferPrivateRemote::setMaximumQueueDepthForTrackID):
(WebKit::SourceBufferPrivateRemote::MessageReceiver::MessageReceiver):
(WebKit::SourceBufferPrivateRemote::takeOwnershipOfMemory): Deleted.
(WebKit::SourceBufferPrivateRemote::sourceBufferPrivateDidReceiveInitializationSegment):
Deleted.
(WebKit::SourceBufferPrivateRemote::sourceBufferPrivateHighestPresentationTimestampChanged):
Deleted.
(WebKit::SourceBufferPrivateRemote::sourceBufferPrivateDurationChanged):
Deleted.
(WebKit::SourceBufferPrivateRemote::sourceBufferPrivateBufferedChanged):
Deleted.
(WebKit::SourceBufferPrivateRemote::sourceBufferPrivateDidDropSample): Deleted.
(WebKit::SourceBufferPrivateRemote::sourceBufferPrivateDidReceiveRenderingError):
Deleted.
* Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h:
*
Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemoteMessageReceiver.h:
Renamed from
Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.messages.in.
The current infrastructure doesn't allow to define the IPC protocol in a
private sub-class, so add an alias.
*
Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemoteMessageReceiver.messages.in:
Renamed from
Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.messages.in.
Canonical link: https://commits.webkit.org/274356@main
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes
