dashorst opened a new pull request, #1876: URL: https://github.com/apache/wicket/pull/1876
`XmlPullParser` now reads attributes the way the HTML tokenizer does, so `<a href=foo/bar.html>` keeps `href="foo/bar.html"` instead of turning into `href="foo" bar.html=""` (#1864). Auditing the parser against the HTML specification, with fuzzing and mutation testing, turned up twelve more defects in the same code: attribute names and quoted values, the quote scanning that finds the end of a tag, raw-text detection for `<script>` and `<style>`, CDATA, and two crashes. This PR fixes all of them and documents the parser's contract in its class Javadoc, which until now was one line. Fixes #1864 ## Commits Each commit can be read on its own. The first adds every test, red where the parser was wrong; each later commit fixes one aspect and turns its own tests green. With this order, each test can be seen to fail before its fix. 1. **Document XmlPullParser and test it against the HTML tokenizer.** Class Javadoc, plus `XmlPullParserAttributeTest`, `XmlPullParserRawTextTest`, and new tests in `XmlPullParserTest` and `FullyBufferedReaderTest`. 2. **Find the end of a tag without backslash escapes or recursion.** `FullyBufferedReader.findOutOfQuotes`: `title="Next >>"` was rejected, about ten thousand `>` in one quoted value overflowed the stack, and `\"` counted as an escaped quote although HTML, XML and the attribute parser have no escapes. 3. **Read attribute names and unquoted values the way HTML does** (#1864). Unquoted values run to whitespace. Names such as `(click)`, `[value]`, `#ref` and `on:click|once` are kept as written. Text that doesn't match is no longer skipped silently, and quoted values are no longer trimmed. 4. **Decide on raw text for script and style by element name and type.** Fixes `<style>` without attributes (off by one), custom elements like `<script-loader>` swallowing their content, `type` sniffed by substring (`data-type=`, unquoted, empty, `application/javascript`, JSON), raw text ending at `</scripts`, and a `StringIndexOutOfBoundsException` on `<script type=>`. 5. **Accept quotes in CDATA sections and reject `</>` with a ParseException.** ## Where to look Core: `wicket-core/.../markup/parser/XmlPullParser.java`. Read `parseAttributes` and `isScriptOrDataBlock` alongside the class Javadoc, which states the contract they implement. Then `wicket-util/.../io/FullyBufferedReader.java#findOutOfQuotes`. Tests (can be skimmed): the four test classes. The expected values come from the HTML specification, not from what the code returned. `MetaPattern.STRING` and `VariableAssignmentParser` are deliberately unchanged. `ValueMap` and `CommaSeparatedVariableParser` use them too, and they are public API. The parser now has its own attribute tokenizer instead. ## Behaviour changes No public API changes. Templates that parsed before may now parse differently: - Unquoted values containing `/`, `:`, `?` and the like, and framework attribute names (Angular, Vue, Alpine, Svelte, htmx), are now rendered as written in component tags and in tags changed by filters. - Whitespace around a quoted value is kept: `class=" a "` used to become `class="a"`. - Text in a tag that isn't an attribute follows HTML recovery: `<tag a="1" "junk">` gains an attribute named `"junk"` instead of `junk`. A duplicate attribute is still a `ParseException`, as before. - `\"` inside a quoted value now ends the value. Before, the bracket finder treated it as an escape (WICKET-3773), but the attribute parser didn't. - A `<script>` with an empty type, `application/javascript` or a JSON type is now raw text. It used to be parsed as a template, so a Wicket component inside one is no longer found. Other non-JavaScript types are still templates (WICKET-5288). - A custom element whose name starts with `script` or `style` is now parsed as markup instead of raw text. ## Testing - `mvn clean verify -Pjs-test` is green. - Crash fuzzing: 1.5 million random markup strings, with only a `ParseException` allowed. It found `</>` and `<script type=>`. A seeded 20,000-input version is in `XmlPullParserTest.anyMarkupEitherParsesOrIsRejectedWithParseException`. - Differential fuzzing against jsoup's HTML5 tokenizer, run outside the repository because jsoup is not a dependency: 400,000 generated attribute lists, zero differences after the fix. The only remaining difference was numeric character references without a closing `;` (`	`), which `Strings.unescapeMarkup` does not decode. That is entity decoding rather than tokenizing, so it is left for a separate issue. - PIT on `XmlPullParser` and `FullyBufferedReader` with these tests kills 192 of 210 mutants. The survivors are equivalent mutants, `getString()` of a conditional comment (which nothing reads), a `>` check in `skipUntil` that duplicates what `next()` does, and helpers this PR doesn't touch. Not changed: for a plain text token, `getString()` returns text from the position marker rather than the token alone. Nothing in Wicket calls it. Backporting to `wicket-11.x` / `wicket-10.x` is a separate decision. The behaviour changes above argue for master only. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
