The reverting patch is deployed, my own basic tests seem to confirm that the issue is resolved. Affected pages might require a reparse to display correctly.

Dec 22 2023, 2:21 PM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

ReleaseTaggerBot edited projects for T353920: HTML entity in wikitext are being over parsed, added: MW-1.42-notes (1.42.0-wmf.10; 2023-12-19); removed MW-1.42-notes (1.42.0-wmf.12; 2024-01-02).

Dec 22 2023, 2:00 PM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

Stashbot added a comment to T353920: HTML entity in wikitext are being over parsed.

Mentioned in SAL (#wikimedia-operations) [2023-12-22T13:45:13Z] <reedy@deploy2002> Finished scap: T353920 (duration: 08m 02s)

Dec 22 2023, 1:45 PM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

Stashbot added a comment to T353920: HTML entity in wikitext are being over parsed.

Mentioned in SAL (#wikimedia-operations) [2023-12-22T13:37:11Z] <reedy@deploy2002> Started scap: T353920

Dec 22 2023, 1:37 PM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

gerritbot added a comment to T353920: HTML entity in wikitext are being over parsed.

Change 985033 merged by jenkins-bot:

[mediawiki/core@wmf/1.42.0-wmf.10] Revert "Use Remex for DeduplicateStyles transform"

https://gerrit.wikimedia.org/r/985033

Dec 22 2023, 1:33 PM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

gerritbot added a project to T353920: HTML entity in wikitext are being over parsed: Patch-For-Review.

Dec 22 2023, 1:11 PM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

gerritbot added a comment to T353920: HTML entity in wikitext are being over parsed.

Change 985033 had a related patch set uploaded (by Reedy; author: Isabelle Hurbain-Palatin):

[mediawiki/core@wmf/1.42.0-wmf.10] Revert "Use Remex for DeduplicateStyles transform"

https://gerrit.wikimedia.org/r/985033

Dec 22 2023, 1:11 PM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

ReleaseTaggerBot added a project to T353920: HTML entity in wikitext are being over parsed: MW-1.42-notes (1.42.0-wmf.12; 2024-01-02).

Dec 22 2023, 12:00 PM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

Maintenance_bot removed a project from T353920: HTML entity in wikitext are being over parsed: Patch-For-Review.

Dec 22 2023, 11:30 AM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

gerritbot added a comment to T353920: HTML entity in wikitext are being over parsed.

Change 985120 merged by jenkins-bot:

[mediawiki/core@master] Revert "Use Remex for DeduplicateStyles transform"

https://gerrit.wikimedia.org/r/985120

Dec 22 2023, 11:03 AM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

ihurbain added a comment to T353920: HTML entity in wikitext are being over parsed.

Agreed on the fact it should exist; not _entirely_ sure about the fact it would have caught it. Will have a look.

Dec 22 2023, 10:55 AM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

TheDJ added a comment to T353920: HTML entity in wikitext are being over parsed.

We should probably have a parser test with the &ndash; combo btw. I did a quick grep, and couldn't find any test case that had an escaped amp followed by an entity name (which makes sense, cause otherwise we would have caught this of course). But also entity forms in general seem not well guarded. A tests/parser/entities.txt might make sense after this.

Dec 22 2023, 10:52 AM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

gerritbot added a project to T353920: HTML entity in wikitext are being over parsed: Patch-For-Review.

Dec 22 2023, 10:26 AM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

gerritbot added a comment to T353920: HTML entity in wikitext are being over parsed.

Change 985120 had a related patch set uploaded (by Isabelle Hurbain-Palatin; author: Isabelle Hurbain-Palatin):

[mediawiki/core@master] Revert "Use Remex for DeduplicateStyles transform"

https://gerrit.wikimedia.org/r/985120

Dec 22 2023, 10:26 AM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

ihurbain added a comment to T353920: HTML entity in wikitext are being over parsed.

The issue is as follows: instead of going through a regex to do the deduplicating of styles, we pass the whole document through Remex, which apparently at some points decides to interpret &ndash; as −. This is indeed caused by the above patch, which I believe is safe to revert (pages parsed in the meantime may require cache purging for the fix to go through.)
Interestingly, the issue does NOT trigger on parsoid rendering, which I suspect may have something to do with " $options['isParsoidContent'] ?? false" that sets thing to html5format vs not in remex.

Dec 22 2023, 10:22 AM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

ihurbain updated the task description for T353920: HTML entity in wikitext are being over parsed.

Dec 22 2023, 9:59 AM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

ihurbain claimed T353920: HTML entity in wikitext are being over parsed.

Dec 22 2023, 9:47 AM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

TheDJ triaged T353920: HTML entity in wikitext are being over parsed as Unbreak Now! priority.

I'm marking this as UBN.

Dec 22 2023, 9:46 AM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression

Xaosflux added a project to T353920: HTML entity in wikitext are being over parsed: RemexHtml.

Dec 22 2023, 1:07 AM · MW-1.42-notes (1.42.0-wmf.10; 2023-12-19), RemexHtml, MediaWiki-Parser, Regression