Page MenuHomePhabricator

Document how to add languages to Wikibase/Wikidata
Closed, ResolvedPublic3 Estimated Story Points

Description

When working on the subtasks of T332714: add new language codes for Wikidata - Spring 2023, there were multiple occasions where some of us were unsure what steps are needed and in which order. Therefore, we should make sure the process to do so is documented clearly so that we can more easily onboard new engineers to this occasional chore for our team.

Acceptance criteria

  • Add clear and concise documentation related to adding or removing a language to/from Wikibase at https://www.mediawiki.org/wiki/Manual:Adding_and_removing_languages#Wikibase
  • Link any Wikibase-related documentation from README.md on Wikibase.git to ensure easy accessibility.
  • The documentation should cover the following tasks in a step-by-step manner:
    • Adding a monolingual text language code to Wikibase, explaining the availability of the names and an the relation to the CLDR extension.
    • Enabling Wikibase label support by explaining why we need to change both Wikibase.git and config files, what each does, and in what order.
    • Making a language code available for WikibaseLexeme.

Event Timeline

Task Prio Notes:

  • Does not affect end users / production
  • Does not affect monitoring efforts
  • Might affect development efforts
  • Affects onboarding efforts
  • Does not affect additional stakeholders
ItamarWMDE renamed this task from Document how to add languages to Wikibase/Wikidata to [SW] Document how to add languages to Wikibase/Wikidata.May 9 2023, 2:39 PM

Story Writing Notes:

  • Two acceptance criteria at least:
    1. Documentation is created (specify location, probably wiki)
    2. Documentation is linked to in relevant places (specify relevant places, probably dev readme)
ItamarWMDE renamed this task from [SW] Document how to add languages to Wikibase/Wikidata to Document how to add languages to Wikibase/Wikidata.May 10 2023, 12:46 PM
ItamarWMDE updated the task description. (Show Details)

I was recently putting together something like this. :) Here's what I have for how to actually add a code:

Information needed:

For Wikidata:

  • Language code
  • Name in English

For language-data (not needed for Wikidata itself, but recommended for consistency):

  • Native name
  • Script
  • Region(s) used: Worldwide, America, Europe, Middle East, Africa, Asia, Pacific

Adding a code for labels (which will automatically add it for monolingual text and lexemes as well):

In mediawiki-extensions-Wikibase:

In operations-mediawiki-config:

Adding a code only for monolingual text:

In mediawiki-extensions-Wikibase:

Adding a code only for lexemes:

In mediawiki-extensions-WikibaseLexeme:

In all cases:

In mediawiki-extensions-cldr:

In language-data:

If it uses a RTL script:

In mediawiki:

explaining the availability of the names

If you find anyone who truly understands that, do let me know. :)

and an the relation to the CLDR extension.

Wikibase gets localised language names from the CLDR extension. Names provided by CLDR are in the CldrNames directory. We can add things which are missing (or override existing ones) in the LocalNames directory. The CLDR extension falls back to wmgExtraLanguageNames and Names.php, so a name might still be shown for languages not in the CLDR extension if the language is defined in one of those places instead.

Enabling Wikibase label support by explaining why we need to change both Wikibase.git and config files, what each does, and in what order.

Both need to be changed because T273627 is not yet resolved. Most parts of Wikidata fetch languages from one, some parts (see the ticket) still rely on the other. I'm not sure if the order matters, but it doesn't work properly without changing the config so that might be the best one to do first.

Note that for lexemes, while the names on Special:NewLexeme come from the WikibaseLexeme i18n directory, the names given by wbcontentlanguages in the API come from the CLDR extension (which I reported at T322139).

@Nikki this is very informative and helpful. Thank you!

Change 923332 had a related patch set uploaded (by Noa wmde; author: Noa wmde):

[mediawiki/extensions/Wikibase@master] Add a section in the README about adding language codes

https://gerrit.wikimedia.org/r/923332

Change 923335 had a related patch set uploaded (by Noa wmde; author: Noa wmde):

[mediawiki/extensions/WikibaseLexeme@master] Add a section about language code support

https://gerrit.wikimedia.org/r/923335

Change 923332 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Add a section in the README about adding language codes

https://gerrit.wikimedia.org/r/923332

Change 923335 merged by jenkins-bot:

[mediawiki/extensions/WikibaseLexeme@master] Add a section about language code support

https://gerrit.wikimedia.org/r/923335

ItamarWMDE updated the task description. (Show Details)
ItamarWMDE updated the task description. (Show Details)

\o/ Done, thank you!

I linked the documentation from the project description of Language codes as well now.