- Revision
- 292543
- Author
- wenson_hs...@apple.com
- Date
- 2022-04-07 10:22:10 -0700 (Thu, 07 Apr 2022)
Log Message
Adjust and refactor some UA styles and logic for injecting Live Text
https://bugs.webkit.org/show_bug.cgi?id=238912
rdar://91383570
Reviewed by Aditya Keerthi.
Source/WebCore:
Adjust various Live-Text-related UA styles for "text recognition blocks", along with other miscellaneous
adjustments:
- Compute and set the border radius, based on the font size and overall height of the block.
- Add horizontal/vertical padding to text recognition, computed relative to the border radius.
- Allow hyphenation in blocks.
- Only center-align text in blocks if there are fewer than 3 text runs in the block.
- Adjust box shadows, backdrop filter blur radius and line height.
- Specify a `line-height`, such that `line-height` from the host element doesn't erroneously apply to blocks.
- Handle text recognition blocks with newline characters (\n) by injecting line break elements between text.
Test: fast/images/text-recognition/image-overlay-block-with-newlines.html
* dom/ImageOverlay.cpp:
(WebCore::ImageOverlay::updateSubtree):
(WebCore::ImageOverlay::fitElementToQuad):
(WebCore::ImageOverlay::updateWithTextRecognitionResult):
* html/shadow/imageOverlay.css:
(:host(:not(img)) div#image-overlay:-webkit-full-screen-document):
(div.image-overlay-block):
(div.image-overlay-line, .image-overlay-text, div.image-overlay-block):
(div.image-overlay-line, .image-overlay-text):
(.image-overlay-block):
LayoutTests:
Add a layout test to exercise "block"-style Live Text injection, in the case where the injected text contains
newlines. The resulting selected text should preserve the newline.
* fast/images/text-recognition/image-overlay-block-with-newlines-expected.txt: Added.
* fast/images/text-recognition/image-overlay-block-with-newlines.html: Added.
Modified Paths
Added Paths
Diff
Modified: trunk/LayoutTests/ChangeLog (292542 => 292543)
--- trunk/LayoutTests/ChangeLog 2022-04-07 17:13:34 UTC (rev 292542)
+++ trunk/LayoutTests/ChangeLog 2022-04-07 17:22:10 UTC (rev 292543)
@@ -1,3 +1,17 @@
+2022-04-07 Wenson Hsieh <wenson_hs...@apple.com>
+
+ Adjust and refactor some UA styles and logic for injecting Live Text
+ https://bugs.webkit.org/show_bug.cgi?id=238912
+ rdar://91383570
+
+ Reviewed by Aditya Keerthi.
+
+ Add a layout test to exercise "block"-style Live Text injection, in the case where the injected text contains
+ newlines. The resulting selected text should preserve the newline.
+
+ * fast/images/text-recognition/image-overlay-block-with-newlines-expected.txt: Added.
+ * fast/images/text-recognition/image-overlay-block-with-newlines.html: Added.
+
2022-04-07 Alan Bujtas <za...@apple.com>
Fix the expected failure type.
Added: trunk/LayoutTests/fast/images/text-recognition/image-overlay-block-with-newlines-expected.txt (0 => 292543)
--- trunk/LayoutTests/fast/images/text-recognition/image-overlay-block-with-newlines-expected.txt (rev 0)
+++ trunk/LayoutTests/fast/images/text-recognition/image-overlay-block-with-newlines-expected.txt 2022-04-07 17:22:10 UTC (rev 292543)
@@ -0,0 +1,5 @@
+PASS getSelection().toString() is "Hello\nworld"
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
Added: trunk/LayoutTests/fast/images/text-recognition/image-overlay-block-with-newlines.html (0 => 292543)
--- trunk/LayoutTests/fast/images/text-recognition/image-overlay-block-with-newlines.html (rev 0)
+++ trunk/LayoutTests/fast/images/text-recognition/image-overlay-block-with-newlines.html 2022-04-07 17:22:10 UTC (rev 292543)
@@ -0,0 +1,31 @@
+<!DOCTYPE html>
+<html>
+<head>
+<script src=""
+<style>
+body, html {
+ margin: 0;
+}
+</style>
+</head>
+<body>
+<img src=""
+<script>
+addEventListener("load", () => {
+ let image = document.querySelector("img");
+ internals.installImageOverlay(image, [], [
+ {
+ topLeft : new DOMPointReadOnly(0.1, 0.1),
+ topRight : new DOMPointReadOnly(0.4, 0.1),
+ bottomRight : new DOMPointReadOnly(0.4, 0.4),
+ bottomLeft : new DOMPointReadOnly(0.1, 0.4),
+ text : "Hello\nworld",
+ }
+ ]);
+ const overlay = internals.shadowRoot(image).getElementById("image-overlay");
+ getSelection().selectAllChildren(overlay);
+ shouldBeEqualToString("getSelection().toString()", "Hello\nworld");
+});
+</script>
+</body>
+</html>
\ No newline at end of file
Modified: trunk/Source/WebCore/ChangeLog (292542 => 292543)
--- trunk/Source/WebCore/ChangeLog 2022-04-07 17:13:34 UTC (rev 292542)
+++ trunk/Source/WebCore/ChangeLog 2022-04-07 17:22:10 UTC (rev 292543)
@@ -1,3 +1,35 @@
+2022-04-07 Wenson Hsieh <wenson_hs...@apple.com>
+
+ Adjust and refactor some UA styles and logic for injecting Live Text
+ https://bugs.webkit.org/show_bug.cgi?id=238912
+ rdar://91383570
+
+ Reviewed by Aditya Keerthi.
+
+ Adjust various Live-Text-related UA styles for "text recognition blocks", along with other miscellaneous
+ adjustments:
+
+ - Compute and set the border radius, based on the font size and overall height of the block.
+ - Add horizontal/vertical padding to text recognition, computed relative to the border radius.
+ - Allow hyphenation in blocks.
+ - Only center-align text in blocks if there are fewer than 3 text runs in the block.
+ - Adjust box shadows, backdrop filter blur radius and line height.
+ - Specify a `line-height`, such that `line-height` from the host element doesn't erroneously apply to blocks.
+ - Handle text recognition blocks with newline characters (\n) by injecting line break elements between text.
+
+ Test: fast/images/text-recognition/image-overlay-block-with-newlines.html
+
+ * dom/ImageOverlay.cpp:
+ (WebCore::ImageOverlay::updateSubtree):
+ (WebCore::ImageOverlay::fitElementToQuad):
+ (WebCore::ImageOverlay::updateWithTextRecognitionResult):
+ * html/shadow/imageOverlay.css:
+ (:host(:not(img)) div#image-overlay:-webkit-full-screen-document):
+ (div.image-overlay-block):
+ (div.image-overlay-line, .image-overlay-text, div.image-overlay-block):
+ (div.image-overlay-line, .image-overlay-text):
+ (.image-overlay-block):
+
2022-04-07 Lauro Moura <lmo...@igalia.com>
Unreviewed, non-unified build fixes
Modified: trunk/Source/WebCore/dom/ImageOverlay.cpp (292542 => 292543)
--- trunk/Source/WebCore/dom/ImageOverlay.cpp 2022-04-07 17:13:34 UTC (rev 292542)
+++ trunk/Source/WebCore/dom/ImageOverlay.cpp 2022-04-07 17:22:10 UTC (rev 292543)
@@ -358,8 +358,15 @@
}
for (size_t index = 0; index < result.blocks.size(); ++index) {
- if (result.blocks[index].text != elements.blocks[index]->textContent())
- return false;
+ auto textContentByLine = result.blocks[index].text.split(newlineCharacter);
+ size_t lineIndex = 0;
+ for (auto& text : childrenOfType<Text>(elements.blocks[index])) {
+ if (textContentByLine.size() <= lineIndex)
+ return false;
+
+ if (textContentByLine[lineIndex++].stripWhiteSpace() != text.wholeText().stripWhiteSpace())
+ return false;
+ }
}
return true;
@@ -428,8 +435,18 @@
for (auto& block : result.blocks) {
auto blockContainer = HTMLDivElement::create(document.get());
blockContainer->classList().add(imageOverlayBlockClass());
+ auto lines = block.text.split(newlineCharacter);
+ for (auto& textContent : lines) {
+ if (blockContainer->hasChildNodes())
+ blockContainer->appendChild(HTMLBRElement::create(document.get()));
+ blockContainer->appendChild(Text::create(document.get(), textContent));
+ }
+
+ constexpr auto maxLineCountForCenterAlignedText = 2;
+ if (lines.size() > maxLineCountForCenterAlignedText)
+ blockContainer->setInlineStyleProperty(CSSPropertyTextAlign, CSSValueStart);
+
elements.root->appendChild(blockContainer);
- blockContainer->appendChild(Text::create(document.get(), makeString('\n', block.text)));
elements.blocks.uncheckedAppend(WTFMove(blockContainer));
}
@@ -445,11 +462,13 @@
return elements;
}
-static RotatedRect fitElementToQuad(HTMLElement& container, const FloatQuad& quad)
+enum class ConstrainHeight : bool { No, Yes };
+static RotatedRect fitElementToQuad(HTMLElement& container, const FloatQuad& quad, ConstrainHeight constrainHeight = ConstrainHeight::Yes)
{
auto bounds = rotatedBoundingRectWithMinimumAngleOfRotation(quad, 0.01);
container.setInlineStyleProperty(CSSPropertyWidth, bounds.size.width(), CSSUnitType::CSS_PX);
- container.setInlineStyleProperty(CSSPropertyHeight, bounds.size.height(), CSSUnitType::CSS_PX);
+ if (constrainHeight == ConstrainHeight::Yes)
+ container.setInlineStyleProperty(CSSPropertyHeight, bounds.size.height(), CSSUnitType::CSS_PX);
container.setInlineStyleProperty(CSSPropertyTransform, makeString(
"translate("_s,
std::round(bounds.center.x() - (bounds.size.width() / 2)), "px, "_s,
@@ -605,6 +624,17 @@
Vector<FontSizeAdjustmentState> elementsToAdjust;
elementsToAdjust.reserveInitialCapacity(result.blocks.size());
+ auto setInlineStylesForBlock = [&](HTMLElement& block, float scale, float targetHeight) {
+ float fontSize = scale * targetHeight;
+ float borderRadius = fontSize / 5 + (targetHeight - fontSize) / 50;
+ block.setInlineStyleProperty(CSSPropertyFontSize, fontSize, CSSUnitType::CSS_PX);
+ block.setInlineStyleProperty(CSSPropertyBorderRadius, makeString(borderRadius, "px"_s));
+ block.setInlineStyleProperty(CSSPropertyPaddingLeft, 2 * borderRadius, CSSUnitType::CSS_PX);
+ block.setInlineStyleProperty(CSSPropertyPaddingRight, 2 * borderRadius, CSSUnitType::CSS_PX);
+ block.setInlineStyleProperty(CSSPropertyPaddingTop, borderRadius, CSSUnitType::CSS_PX);
+ block.setInlineStyleProperty(CSSPropertyPaddingBottom, borderRadius, CSSUnitType::CSS_PX);
+ };
+
ASSERT(result.blocks.size() == elements.blocks.size());
for (size_t index = 0; index < result.blocks.size(); ++index) {
auto& block = result.blocks[index];
@@ -612,8 +642,8 @@
continue;
auto blockContainer = elements.blocks[index];
- auto bounds = fitElementToQuad(blockContainer.get(), convertToContainerCoordinates(block.normalizedQuad));
- blockContainer->setInlineStyleProperty(CSSPropertyFontSize, initialScaleForFontSize * bounds.size.height(), CSSUnitType::CSS_PX);
+ auto bounds = fitElementToQuad(blockContainer.get(), convertToContainerCoordinates(block.normalizedQuad), ConstrainHeight::No);
+ setInlineStylesForBlock(blockContainer.get(), initialScaleForFontSize, bounds.size.height());
elementsToAdjust.uncheckedAppend({ WTFMove(blockContainer), bounds.size });
}
@@ -622,24 +652,16 @@
document->updateLayoutIgnorePendingStylesheets();
for (auto& state : elementsToAdjust) {
- RefPtr textNode = state.container->firstChild();
- if (!is<Text>(textNode)) {
- ASSERT_NOT_REACHED();
- state.mayRequireAdjustment = false;
+ auto* box = state.container->renderBox();
+ if (!box)
continue;
- }
- auto* textRenderer = downcast<Text>(*textNode).renderer();
- if (!textRenderer) {
- ASSERT_NOT_REACHED();
- state.mayRequireAdjustment = false;
- continue;
- }
-
- auto currentScore = (textRenderer->linesBoundingBox().size() / state.targetSize).maxDimension();
- if (currentScore < minTargetScore)
+ auto targetHeight = state.targetSize.height();
+ auto currentScore = box->contentHeight() / targetHeight;
+ bool hasHorizontalOverflow = box->hasHorizontalOverflow();
+ if (currentScore < minTargetScore && !hasHorizontalOverflow)
state.minScale = state.scale;
- else if (currentScore > maxTargetScore)
+ else if (currentScore > maxTargetScore || hasHorizontalOverflow)
state.maxScale = state.scale;
else {
state.mayRequireAdjustment = false;
@@ -647,7 +669,7 @@
}
state.scale = (state.minScale + state.maxScale) / 2;
- state.container->setInlineStyleProperty(CSSPropertyFontSize, state.targetSize.height() * state.scale, CSSUnitType::CSS_PX);
+ setInlineStylesForBlock(state.container.get(), state.scale, targetHeight);
}
elementsToAdjust.removeAllMatching([](auto& state) {
Modified: trunk/Source/WebCore/html/shadow/imageOverlay.css (292542 => 292543)
--- trunk/Source/WebCore/html/shadow/imageOverlay.css 2022-04-07 17:13:34 UTC (rev 292542)
+++ trunk/Source/WebCore/html/shadow/imageOverlay.css 2022-04-07 17:22:10 UTC (rev 292543)
@@ -34,6 +34,10 @@
font-family: system-ui;
}
+:host(:not(img)) div#image-overlay:-webkit-full-screen-document {
+ display: none;
+}
+
div.image-overlay-line {
white-space: nowrap;
line-height: 100%;
@@ -46,8 +50,7 @@
div.image-overlay-block {
background-color: rgba(255, 255, 255, 0.75);
- border-radius: calc(clamp(2px, 0.1em, 12px));
- box-shadow: rgba(100, 100, 100, 0.2) 3px 4px 8px 4px;
+ box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2), 0 1px 2px rgba(0, 0, 0, 0.3);
color: rgb(90, 90, 90);
font-weight: bold;
display: flex;
@@ -54,14 +57,25 @@
justify-content: center;
align-content: center;
flex-direction: column;
- -webkit-backdrop-filter: blur(8px);
+ -webkit-backdrop-filter: blur(40px);
+ -webkit-hyphens: auto;
+ line-height: 1.2em;
+ box-sizing: border-box;
+ white-space: normal;
}
div.image-overlay-line, .image-overlay-text, div.image-overlay-block {
position: absolute;
+}
+
+div.image-overlay-line, .image-overlay-text {
overflow: hidden;
}
+.image-overlay-block {
+ overflow-x: hidden;
+}
+
.image-overlay-text::selection {
color: transparent;
background-color: highlight;
