nicoscandolo opened a new issue, #1925: URL: https://github.com/apache/stormcrawler/issues/1925
When extracting text from HTML, `JSoupTextExtractor` does not insert a space between a block-level element and an immediately following inline element. For example, given this HTML from https://www.acai-island.com/contact: ```html <div> <h3>Email</h3> <a href="mailto:[email protected]";>[email protected]</a> </div> ``` The extracted text is `[email protected]` instead of `Email [email protected]`. The issue is in the `tail()` method of the `NodeVisitor` inside `JSoupTextExtractor.text()`. Currently, a space is only appended after a block element when the next sibling is a `TextNode`: ```java if (element.isBlock() && (node.nextSibling() instanceof TextNode) && !lastCharIsWhitespace(accum)) { accum.append(' '); } ``` This misses cases where the next sibling is an inline `Element` (like `<a>`, `<span>`, etc.), which causes the text content of the block and the inline element to be concatenated without any separator. This also affects other common patterns like `<div>Phone</div><span>(631) 656-0088</span>` producing `Phone(631) 656-0088`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
