- Revision
- 130810
- Author
- jon...@apple.com
- Date
- 2012-10-09 14:23:32 -0700 (Tue, 09 Oct 2012)
Log Message
[WK2] Have plugins render offscreen to capture snapshot
https://bugs.webkit.org/show_bug.cgi?id=98326
<rdar://problem/12426658>
Reviewed by Simon Fraser.
Source/WebCore:
Change updateSnapshot() to use a PassRefPtr<Image> instead of Image*. WebKit2 ultimately
hands the image off to RenderSnapshottedPlugin. A CachedImage instance then manages the
lifetime of the Image.
* html/HTMLPlugInElement.h:
(WebCore::HTMLPlugInElement::updateSnapshot):
* html/HTMLPlugInImageElement.cpp:
(WebCore::HTMLPlugInImageElement::updateSnapshot):
* html/HTMLPlugInImageElement.h:
(HTMLPlugInImageElement):
* rendering/RenderSnapshottedPlugIn.cpp:
(WebCore::RenderSnapshottedPlugIn::updateSnapshot): Updated to use the PassRefPtr<Image>
pointer.
* rendering/RenderSnapshottedPlugIn.h:
(RenderSnapshottedPlugIn):
Source/WebKit2:
Keep the plugin rendering without compositing to easily grab the snapshot.
PluginView now has two variables referring to snapshots, but are unrelated. The timer
is used to capture a snapshot that acts as a poster for a plugin. The ShareableBitmap
variable m_snapshot is used whenever the plugin paints in software, to avoid side effects
should the plugin run JS during painting.
* WebProcess/Plugins/PluginView.cpp:
(WebKit): Add a named constant for the time delay before a snapshot is taken.
(WebKit::PluginView::PluginView): Initialize a 3-second timer to get the snapshot.
(WebKit::PluginView::~PluginView): Refactor part of the destructor code out into
destroyPluginAndReset() for reuse.
(WebKit::PluginView::destroyPluginAndReset): Contains part of the destructor code.
In addition to destroying the plugin, the destructor cancels pending loads and streams.
(WebKit::PluginView::didInitializePlugin): If the plugin is in a state where it needs
to generate or display a poster, don't setup the compositing layer and start the timer.
(WebKit::PluginView::paint): Avoid painting if the plugin is not running.
(WebKit::PluginView::invalidateRect): Avoid painting if the plugin is not running.
(WebKit::PluginView::isAcceleratedCompositingEnabled): Don't enable accelerated compositing
until the plugin is running.
(WebKit::PluginView::pluginSnapshotTimerFired): When the timer fires, get a snapshot, generate
an Image that WebCore can render, and destroy the plugin.
Rename m_snapshot to m_transientPaintingSnapshot.
* WebProcess/Plugins/PluginView.h:
* WebProcess/Plugins/PluginView.cpp:
(WebKit::PluginView::paint):
(WebKit::PluginView::notifyWidget):
(WebKit::PluginView::pluginSnapshotTimerFired):
Modified Paths
Diff
Modified: trunk/Source/WebCore/ChangeLog (130809 => 130810)
--- trunk/Source/WebCore/ChangeLog 2012-10-09 21:20:35 UTC (rev 130809)
+++ trunk/Source/WebCore/ChangeLog 2012-10-09 21:23:32 UTC (rev 130810)
@@ -1,3 +1,27 @@
+2012-10-09 Jon Lee <jon...@apple.com>
+
+ [WK2] Have plugins render offscreen to capture snapshot
+ https://bugs.webkit.org/show_bug.cgi?id=98326
+ <rdar://problem/12426658>
+
+ Reviewed by Simon Fraser.
+
+ Change updateSnapshot() to use a PassRefPtr<Image> instead of Image*. WebKit2 ultimately
+ hands the image off to RenderSnapshottedPlugin. A CachedImage instance then manages the
+ lifetime of the Image.
+
+ * html/HTMLPlugInElement.h:
+ (WebCore::HTMLPlugInElement::updateSnapshot):
+ * html/HTMLPlugInImageElement.cpp:
+ (WebCore::HTMLPlugInImageElement::updateSnapshot):
+ * html/HTMLPlugInImageElement.h:
+ (HTMLPlugInImageElement):
+ * rendering/RenderSnapshottedPlugIn.cpp:
+ (WebCore::RenderSnapshottedPlugIn::updateSnapshot): Updated to use the PassRefPtr<Image>
+ pointer.
+ * rendering/RenderSnapshottedPlugIn.h:
+ (RenderSnapshottedPlugIn):
+
2012-10-09 Adam Barth <aba...@webkit.org>
Unreviewed. Move this file to where the build systems think it should
Modified: trunk/Source/WebCore/html/HTMLPlugInElement.h (130809 => 130810)
--- trunk/Source/WebCore/html/HTMLPlugInElement.h 2012-10-09 21:20:35 UTC (rev 130809)
+++ trunk/Source/WebCore/html/HTMLPlugInElement.h 2012-10-09 21:23:32 UTC (rev 130810)
@@ -55,7 +55,7 @@
};
DisplayState displayState() const { return m_displayState; }
void setDisplayState(DisplayState state) { m_displayState = state; }
- virtual void updateSnapshot(Image*) { }
+ virtual void updateSnapshot(PassRefPtr<Image>) { }
#if ENABLE(NETSCAPE_PLUGIN_API)
NPObject* getNPObject();
Modified: trunk/Source/WebCore/html/HTMLPlugInImageElement.cpp (130809 => 130810)
--- trunk/Source/WebCore/html/HTMLPlugInImageElement.cpp 2012-10-09 21:20:35 UTC (rev 130809)
+++ trunk/Source/WebCore/html/HTMLPlugInImageElement.cpp 2012-10-09 21:23:32 UTC (rev 130810)
@@ -261,7 +261,7 @@
static_cast<HTMLPlugInImageElement*>(n)->updateWidgetIfNecessary();
}
-void HTMLPlugInImageElement::updateSnapshot(Image* image)
+void HTMLPlugInImageElement::updateSnapshot(PassRefPtr<Image> image)
{
if (displayState() > WaitingForSnapshot || !renderer()->isSnapshottedPlugIn())
return;
Modified: trunk/Source/WebCore/html/HTMLPlugInImageElement.h (130809 => 130810)
--- trunk/Source/WebCore/html/HTMLPlugInImageElement.h 2012-10-09 21:20:35 UTC (rev 130809)
+++ trunk/Source/WebCore/html/HTMLPlugInImageElement.h 2012-10-09 21:23:32 UTC (rev 130810)
@@ -90,7 +90,7 @@
void updateWidgetIfNecessary();
virtual bool useFallbackContent() const { return false; }
- virtual void updateSnapshot(Image*) OVERRIDE;
+ virtual void updateSnapshot(PassRefPtr<Image>) OVERRIDE;
bool m_needsWidgetUpdate;
bool m_shouldPreferPlugInsForImages;
Modified: trunk/Source/WebCore/rendering/RenderSnapshottedPlugIn.cpp (130809 => 130810)
--- trunk/Source/WebCore/rendering/RenderSnapshottedPlugIn.cpp 2012-10-09 21:20:35 UTC (rev 130809)
+++ trunk/Source/WebCore/rendering/RenderSnapshottedPlugIn.cpp 2012-10-09 21:23:32 UTC (rev 130810)
@@ -53,13 +53,13 @@
return static_cast<HTMLPlugInImageElement*>(node());
}
-void RenderSnapshottedPlugIn::updateSnapshot(Image* image)
+void RenderSnapshottedPlugIn::updateSnapshot(PassRefPtr<Image> image)
{
// Zero-size plugins will have no image.
if (!image)
return;
- m_snapshotResource->setCachedImage(new CachedImage(image));
+ m_snapshotResource->setCachedImage(new CachedImage(image.get()));
repaint();
}
Modified: trunk/Source/WebCore/rendering/RenderSnapshottedPlugIn.h (130809 => 130810)
--- trunk/Source/WebCore/rendering/RenderSnapshottedPlugIn.h 2012-10-09 21:20:35 UTC (rev 130809)
+++ trunk/Source/WebCore/rendering/RenderSnapshottedPlugIn.h 2012-10-09 21:23:32 UTC (rev 130810)
@@ -40,7 +40,7 @@
RenderSnapshottedPlugIn(HTMLPlugInImageElement*);
virtual ~RenderSnapshottedPlugIn();
- void updateSnapshot(Image*);
+ void updateSnapshot(PassRefPtr<Image>);
private:
HTMLPlugInImageElement* plugInImageElement() const;
Modified: trunk/Source/WebKit2/ChangeLog (130809 => 130810)
--- trunk/Source/WebKit2/ChangeLog 2012-10-09 21:20:35 UTC (rev 130809)
+++ trunk/Source/WebKit2/ChangeLog 2012-10-09 21:23:32 UTC (rev 130810)
@@ -1,3 +1,41 @@
+2012-10-09 Jon Lee <jon...@apple.com>
+
+ [WK2] Have plugins render offscreen to capture snapshot
+ https://bugs.webkit.org/show_bug.cgi?id=98326
+ <rdar://problem/12426658>
+
+ Reviewed by Simon Fraser.
+
+ Keep the plugin rendering without compositing to easily grab the snapshot.
+
+ PluginView now has two variables referring to snapshots, but are unrelated. The timer
+ is used to capture a snapshot that acts as a poster for a plugin. The ShareableBitmap
+ variable m_snapshot is used whenever the plugin paints in software, to avoid side effects
+ should the plugin run JS during painting.
+
+ * WebProcess/Plugins/PluginView.cpp:
+ (WebKit): Add a named constant for the time delay before a snapshot is taken.
+ (WebKit::PluginView::PluginView): Initialize a 3-second timer to get the snapshot.
+ (WebKit::PluginView::~PluginView): Refactor part of the destructor code out into
+ destroyPluginAndReset() for reuse.
+ (WebKit::PluginView::destroyPluginAndReset): Contains part of the destructor code.
+ In addition to destroying the plugin, the destructor cancels pending loads and streams.
+ (WebKit::PluginView::didInitializePlugin): If the plugin is in a state where it needs
+ to generate or display a poster, don't setup the compositing layer and start the timer.
+ (WebKit::PluginView::paint): Avoid painting if the plugin is not running.
+ (WebKit::PluginView::invalidateRect): Avoid painting if the plugin is not running.
+ (WebKit::PluginView::isAcceleratedCompositingEnabled): Don't enable accelerated compositing
+ until the plugin is running.
+ (WebKit::PluginView::pluginSnapshotTimerFired): When the timer fires, get a snapshot, generate
+ an Image that WebCore can render, and destroy the plugin.
+
+ Rename m_snapshot to m_transientPaintingSnapshot.
+ * WebProcess/Plugins/PluginView.h:
+ * WebProcess/Plugins/PluginView.cpp:
+ (WebKit::PluginView::paint):
+ (WebKit::PluginView::notifyWidget):
+ (WebKit::PluginView::pluginSnapshotTimerFired):
+
2012-10-09 Rik Cabanier <caban...@adobe.com>
Add missing compile flag for compositing
Modified: trunk/Source/WebKit2/WebProcess/Plugins/PluginView.cpp (130809 => 130810)
--- trunk/Source/WebKit2/WebProcess/Plugins/PluginView.cpp 2012-10-09 21:20:35 UTC (rev 130809)
+++ trunk/Source/WebKit2/WebProcess/Plugins/PluginView.cpp 2012-10-09 21:23:32 UTC (rev 130810)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2010 Apple Inc. All rights reserved.
+ * Copyright (C) 2010, 2012 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -67,6 +67,8 @@
namespace WebKit {
+static const double pluginSnapshotTimerDelay = 3;
+
class PluginView::URLRequest : public RefCounted<URLRequest> {
public:
static PassRefPtr<PluginView::URLRequest> create(uint64_t requestID, const FrameLoadRequest& request, bool allowPopups)
@@ -268,6 +270,7 @@
, m_npRuntimeObjectMap(this)
#endif
, m_manualStreamState(StreamStateInitial)
+ , m_pluginSnapshotTimer(this, &PluginView::pluginSnapshotTimerFired, pluginSnapshotTimerDelay)
{
m_webPage->addPluginView(this);
}
@@ -282,6 +285,15 @@
if (m_isWaitingUntilMediaCanStart)
m_pluginElement->document()->removeMediaCanStartListener(this);
+ destroyPluginAndReset();
+
+ // Null out the plug-in element explicitly so we'll crash earlier if we try to use
+ // the plug-in view after it's been destroyed.
+ m_pluginElement = nullptr;
+}
+
+void PluginView::destroyPluginAndReset()
+{
// Cancel all pending frame loads.
for (FrameLoadMap::iterator it = m_pendingFrameLoads.begin(), end = m_pendingFrameLoads.end(); it != end; ++it)
it->key->setLoadListener(0);
@@ -302,10 +314,6 @@
#endif
cancelAllStreams();
-
- // Null out the plug-in element explicitly so we'll crash earlier if we try to use
- // the plug-in view after it's been destroyed.
- m_pluginElement = nullptr;
}
Frame* PluginView::frame() const
@@ -513,7 +521,9 @@
redeliverManualStream();
#if PLATFORM(MAC)
- if (m_plugin->pluginLayer()) {
+ if (m_pluginElement->displayState() < HTMLPlugInElement::Playing)
+ m_pluginSnapshotTimer.restart();
+ else if (m_plugin->pluginLayer()) {
if (frame()) {
frame()->view()->enterCompositingMode();
m_pluginElement->setNeedsStyleRecalc(SyntheticStyleChange);
@@ -643,7 +653,7 @@
void PluginView::paint(GraphicsContext* context, const IntRect& /*dirtyRect*/)
{
- if (!m_plugin || !m_isInitialized)
+ if (!m_plugin || !m_isInitialized || m_pluginElement->displayState() < HTMLPlugInElement::Playing)
return;
if (context->paintingDisabled()) {
@@ -658,8 +668,8 @@
if (paintRect.isEmpty())
return;
- if (m_snapshot) {
- m_snapshot->paint(*context, contentsScaleFactor(), frameRect().location(), m_snapshot->bounds());
+ if (m_transientPaintingSnapshot) {
+ m_transientPaintingSnapshot->paint(*context, contentsScaleFactor(), frameRect().location(), m_transientPaintingSnapshot->bounds());
return;
}
@@ -736,10 +746,10 @@
switch (notification) {
case WillPaintFlattened:
if (m_plugin && m_isInitialized)
- m_snapshot = m_plugin->snapshot();
+ m_transientPaintingSnapshot = m_plugin->snapshot();
break;
case DidPaintFlattened:
- m_snapshot = nullptr;
+ m_transientPaintingSnapshot = nullptr;
break;
}
}
@@ -1021,6 +1031,9 @@
return;
#endif
+ if (m_pluginElement->displayState() < HTMLPlugInElement::Playing)
+ return;
+
RenderBoxModelObject* renderer = toRenderBoxModelObject(m_pluginElement->renderer());
if (!renderer)
return;
@@ -1176,6 +1189,8 @@
if (!settings)
return false;
+ if (m_pluginElement->displayState() < HTMLPlugInElement::Playing)
+ return false;
return settings->acceleratedCompositingEnabled();
}
@@ -1364,4 +1379,19 @@
}
#endif
+void PluginView::pluginSnapshotTimerFired(DeferrableOneShotTimer<PluginView>* timer)
+{
+ ASSERT_UNUSED(timer, timer == &m_pluginSnapshotTimer);
+ ASSERT(m_plugin);
+
+ // Snapshot might be 0 if plugin size is 0x0.
+ RefPtr<ShareableBitmap> snapshot = m_plugin->snapshot();
+ RefPtr<Image> snapshotImage;
+ if (snapshot)
+ snapshotImage = snapshot->createImage();
+ m_pluginElement->updateSnapshot(snapshotImage.release());
+ destroyPluginAndReset();
+ m_plugin = 0;
+}
+
} // namespace WebKit
Modified: trunk/Source/WebKit2/WebProcess/Plugins/PluginView.h (130809 => 130810)
--- trunk/Source/WebKit2/WebProcess/Plugins/PluginView.h 2012-10-09 21:20:35 UTC (rev 130809)
+++ trunk/Source/WebKit2/WebProcess/Plugins/PluginView.h 2012-10-09 21:23:32 UTC (rev 130810)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2010 Apple Inc. All rights reserved.
+ * Copyright (C) 2010, 2012 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -30,11 +30,13 @@
#include "Plugin.h"
#include "PluginController.h"
#include "WebFrame.h"
+#include <WebCore/Image.h>
#include <WebCore/MediaCanStartListener.h>
#include <WebCore/PluginViewBase.h>
#include <WebCore/ResourceError.h>
#include <WebCore/ResourceResponse.h>
#include <WebCore/RunLoop.h>
+#include <WebCore/Timer.h>
#include <wtf/Deque.h>
// FIXME: Eventually this should move to WebCore.
@@ -104,6 +106,8 @@
void redeliverManualStream();
+ void pluginSnapshotTimerFired(WebCore::DeferrableOneShotTimer<PluginView>*);
+
// WebCore::PluginViewBase
#if PLATFORM(MAC)
virtual PlatformLayer* platformLayer() const;
@@ -177,6 +181,7 @@
virtual void didInitializePlugin();
virtual void didFailToInitializePlugin();
+ void destroyPluginAndReset();
// WebFrame::LoadListener
virtual void didFinishLoad(WebFrame*);
@@ -222,7 +227,10 @@
WebCore::ResourceError m_manualStreamError;
RefPtr<WebCore::SharedBuffer> m_manualStreamData;
- RefPtr<ShareableBitmap> m_snapshot;
+ // This snapshot is used to avoid side effects should the plugin run JS during painting.
+ RefPtr<ShareableBitmap> m_transientPaintingSnapshot;
+ // This timer is used when plugin snapshotting is enabled, to capture a plugin placeholder.
+ WebCore::DeferrableOneShotTimer<PluginView> m_pluginSnapshotTimer;
};
} // namespace WebKit
