In case it may have something to do with a particular trait of a lexeme, I have encountered it on L4177 (you'll see my (User:Ijon) recent edits there, after which the display returned to normal (but not before).

notes: purging does not always fix the issue on its own; however, clearing the browser caches does

It just happened to me also (on many lexemes, including L1, L2, L3, L4, L5, L16, L3870, L448884 etc.).
For some context, it appeared on multiple browsers (Firefox, Chrome, Edge) and also when log out.

Apparently the wikibase.desktop RL module is missing for some reason; manually loading it via the console mostly fixes the styles.

And that means I’ve also found a way to reproduce this, reliably AFAICT:

1. Go to L3690 on desktop Test Wikidata
2. Observe that it looks correct
3. Go to purge L3690 on mobile Test Wikidata
4. Confirm purge
5. Go back to desktop version
6. Observe that it looks broken

Which means this is yet another fallout of T324991: Make Wikibase not rely on the ResourceLoader target system.

And I think the only reason this doesn’t affect items and properties is that items and properties use the mobile termbox, and splitting the parser cache by termbox version (here) effectively splits it by desktop vs. mobile as well. But at least for lexemes, and probably in general, I think we need to split the parser cache by desktop/mobile explicitly, at least as long as we continue to have RL modules that should only be loaded on one target despite the ResourceLoader target system having been removed.

<!-- Saved in parser cache with key testwikidatawiki:pcache:idhash:47524-0!termboxVersion=1!wb=3 and timestamp 20230925095702 and revision id 655448. Rendering was triggered because: page-view
 -->

<!-- Saved in parser cache with key testwikidatawiki:pcache:idhash:47524-0!termboxVersion=22!wb=3 and timestamp 20230925095657 and revision id 655448. Rendering was triggered because: page-view
 -->

Alternatively, I suppose we could add $parserOutput->recordOption( 'termboxVersion' ); to LexemeHandler, to make the item/property “solution” work for lexemes as well.

Change 960564 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] Split parser cache by desktop/mobile

https://gerrit.wikimedia.org/r/960564

Note that the proposed patch above won’t immediately fix the issue for already-broken pages, but new instances of the issue should stop happening and old broken pages would fall out of the parser cache within a month (or rather, “temporarily” three weeks). We could accelerate this by proactively purging all lexemes, but at the current number of lexemes (over a million) and purge rate limit (30 per minute), that would take almost a month, by which time the parser cache would have expired anyway – so I’m not sure that’s worth doing.

Quoting from the commit message of the change linked above:

Judging by some local testing, this should not effectively clear the parser cache: as far as I can tell, parser cache contents from before this code is deployed are still used for desktop page views afterwards (and the “Saved in parser cache” HTML comment indicates a cache key without wbMobile), but as soon as a mobile page view comes in, it removes the old parser cache content and adds one with wbMobile=1; the next desktop page view then generates a parser content with wbMobile=0.

I just tested this on mwdebug2002 in production and it behaved the same way there. I put the details in P52867, but basically, on items properties and lexemes, the desktop page views continued to serve the old parser cache content until I made a mobile request, at which point both desktop and mobile page views started to respond with new parser cache entries that had wbMobile in the key.

Of course, this only tests the application-level cache; the HTTP-level cache can’t be tested with mwdebug.

In T344362#9235624, @Lucas_Werkmeister_WMDE wrote:

Quoting from the commit message of the change linked above:

Judging by some local testing, this should not effectively clear the parser cache: as far as I can tell, parser cache contents from before this code is deployed are still used for desktop page views afterwards (and the “Saved in parser cache” HTML comment indicates a cache key without wbMobile), but as soon as a mobile page view comes in, it removes the old parser cache content and adds one with wbMobile=1; the next desktop page view then generates a parser content with wbMobile=0.

I just tested this on mwdebug2002 in production and it behaved the same way there. I put the details in P52867, but basically, on items properties and lexemes, the desktop page views continued to serve the old parser cache content until I made a mobile request, at which point both desktop and mobile page views started to respond with new parser cache entries that had wbMobile in the key.

Of course, this only tests the application-level cache; the HTTP-level cache can’t be tested with mwdebug.

Thank you! But looking at the details, I just realized that there might be a step missing. Did we make sure that Items and Properties did have a mobile parsercache (with mobile Termbox) before deploying the change? If not, then there would be no surprise about it being regenerated.

I had loaded the mobile page before as well, yes – I added some more details in P52867#213725 (that was supposed to be a before+after with curl but I got the server wrong; but at least it shows there was something in the mobile parser cache).

Oh, and I think I forgot to mention the most important part :D I repeated the steps from T344362#9194424 and was unable to reproduce the issue while the patch was applied on mwdebug2002.

In T344362#9235716, @Lucas_Werkmeister_WMDE wrote:

Oh, and I think I forgot to mention the most important part :D I repeated the steps from T344362#9194424 and was unable to reproduce the issue while the patch was applied on mwdebug2002.

Great news!

Now we just need to understand why the parser cache behaves the way it does (regenerate on mobile with the patch), and what the implications are (performance and otherwise).

Now we just need to understand why the parser cache behaves the way it does (regenerate on mobile with the patch), and what the implications are (performance and otherwise).

I think I understand it now. There’s been another revelation in the lab.

In T344362#9235648, @Michael wrote:

looking at the details, I just realized that there might be a step missing. Did we make sure that Items and Properties did have a mobile parsercache (with mobile Termbox) before deploying the change? If not, then there would be no surprise about it being regenerated.

In T344362#9235703, @Lucas_Werkmeister_WMDE wrote:

I had loaded the mobile page before as well, yes – I added some more details in P52867#213725 (that was supposed to be a before+after with curl but I got the server wrong; but at least it shows there was something in the mobile parser cache).

Michael was right. As you can see in that paste, I had loaded the mobile page before on mwdebug1001, in eqiad – but I did all my testing on mwdebug2002, in codfw. eqiad and codfw have separate parser caches (AFAICT), so as far as mwdebug2002 was concerned, the parser cache for the items and properties was blank. And there is indeed no surprise about it being regenerated then.

The reason that regenerating the mobile parser cache also invalidated the desktop parser cache is that, when the parser cache is saved, it doesn’t just add a new pcache:idhash entry to the parser cache (example: my local wiki has wiki1:pcache:idhash:3231-0!termboxVersion=1!wb=3!en, wiki1:pcache:idhash:3231-0!termboxVersion=22!wb=3!en, wiki1:pcache:idhash:3231-0!termboxVersion=1!wb=3!wbMobile=0!en, and more, all for the same page + revision), but also adds or updates the idoptions entry which records which parser options were used. Before, it may look like this:

MariaDB [wiki1]> SELECT value FROM objectcache WHERE keyname = 'wiki1:pcache:idoptions:3231'\G
*************************** 1. row ***************************
value: s:180:"{"ParseUsedOptions":{"userlang":true,"wb":true,"termboxVersion":true},"CacheExpiry":86400,"CacheTime":"20231013133621","CacheRevisionId":6778,"_type_":"CacheTime","_complex_":true}";
1 row in set (0.001 sec)

Whereas after the mobile parse, it may look like this instead:

MariaDB [wiki1]> SELECT value FROM objectcache WHERE keyname = 'wiki1:pcache:idoptions:3231'\G
*************************** 1. row ***************************
value: s:196:"{"ParseUsedOptions":{"wbMobile":true,"userlang":true,"wb":true,"termboxVersion":true},"CacheExpiry":86400,"CacheTime":"20231013145932","CacheRevisionId":6778,"_type_":"CacheTime","_complex_":true}";
1 row in set (0.001 sec)

The wbMobile option is now recorded as used, which is why the next desktop page view then also regenerates the parser cache (since the existing parser cache entry for the desktop doesn’t have this option).

If you actually make sure that both desktop and mobile are in the parser cache, then deploying the code change won’t cause either type of page view to recalculate anything – they’ll both continue using the old parser cache entries. At least according to some testing on localhost; I don’t really want to repeat the test on mwdebug, to be honest. (Also, note that this is only about items and properties, where the parser cache is already split between desktop and mobile due to the termboxVersion. Lexemes have another, different problem with the parser cache, see T348872.)

Oh, and some minor notes for debugging this locally:

• If you don’t have a working termbox SSR service, you’ll want to comment out the call to $parserOutput->updateCacheExpiry( 0 ); in FullEntityParserOutputGenerator::addHtmlToParserOutput(), otherwise mobile item or property page views won’t end up in the parser cache due to the broken termbox SSR. (If you’re switching between branches to try out parser cache behavior with vs. without the patch, make sure to do this on both branches.)
• The parser cache is easier to understand when it’s in the database compared to e.g. memcached; I recommend $wgParserCacheType = CACHE_DB. Then you can inspect the objectcache table using the mysql.php maintenance script.
• The parser cache is even easier to understand when its contents (objectcache.value) are not compressed; consider forcing $this->hasZlib to false in the SqlBagOStuff constructor. (You’ll want to TRUNCATE TABLE objectcache; after doing this, since it renders all existing cache entries unusable.)

Change 960564 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Split parser cache by desktop/mobile

https://gerrit.wikimedia.org/r/960564

Should roll out with this week’s train.