summaryrefslogtreecommitdiffstats
path: root/util/locale_database
Commit message (Collapse)AuthorAgeFilesLines
...
* QLocaleXML: use attributes in the zone alias tableEdward Welbourne2024-07-152-8/+6
| | | | | | | | | Replacing elements for the alias and IANA ID with attribute makes the table more compact, albeit the ComodRivadavia like is a little long. (Some existing msLandZones/ianaids lines are longer, though.) Change-Id: Iab2b55a21857402ad7c863ef33abd241f1d58a8d Reviewed-by: Mate Barany <[email protected]>
* QLocaleXML: use attributes for likely subtag tag partsEdward Welbourne2024-07-152-14/+17
| | | | | | | | | | This makes the likely subtag part of the file more compact. Introduces a QLocaleXmlWriter.asTag() for attribute-only elements; this requires the Spacer to recognize self-closing elements as not increasing the indent needed. Change-Id: I1b73b755f9841617a5c002cf624785321e808d0c Reviewed-by: Mate Barany <[email protected]>
* QLocaleXML: Use enum values instead of names in likely subtag mapEdward Welbourne2024-07-113-24/+35
| | | | | | | | | | | The existing naming lists provide the needed mapping and this prepares the way to move the language, script and territory into the from and to elements as attributes, saving some file-size. It incidentally pushes the mapping to enum values upstream and simplifies the downstream processing. Change-Id: I8f6d2615d52b14d46d1b795539c71f8afdc310ca Reviewed-by: Dennis Oberst <[email protected]>
* QLocaleXML: Omit code forms of locale tagsEdward Welbourne2024-07-112-8/+4
| | | | | | | | | | | | | | These were written (and empty for the Any* enum members) but never read. We, in any case, infer what we need from the enum members, via the languageList, scriptList and territoryList elements. In the process, add a comment between the fromXml() and toXml() methods of Locale to remind those editing the code to also edit the schema describing the XML. Change-Id: Ie5e51f594c2636802eefd8159954105718d9af52 Reviewed-by: Øystein Heskestad <[email protected]> Reviewed-by: Mate Barany <[email protected]>
* Fix naming of timezone data tablesEdward Welbourne2024-07-111-8/+7
| | | | | | | | | | | | | | | | | | Two FIXME comments related to the misnaming of string data tables: a table of space-joined lists of IANA IDs was named ianaIdData, as a result of which a table of single IANA IDs (and some aliases) was named aliasIdData. A field in one struct was an index into the former even though its values were actually single IANA IDs. So rename the list data table to ianaListData, reusing its old name for the former ianaIdData, and transfer the single-ID data from the ID-list table to the single-ID table. Moving that data changed indexing into both string tables and thus all of the data-tables referencing these tables. Task-number: QTBUG-115158 Change-Id: I84165736e91d0bf127f3f9f3b95e9c3060a30c12 Reviewed-by: Mate Barany <[email protected]>
* Update CLDR to v45, adding language KuviEdward Welbourne2024-07-112-2/+5
| | | | | | | | | | | | | | | | | | | This was in fact present in v44, but we overlooked it somehow. The new version also fixes some inconsistencies in the data, that I reported against v44.1; in particular, Tamil no longer claims to override the root AM/PM markers (probably because it uses 24-hour time so doesn't need them). Add the test-file under util to the list of files containing generated content. [ChangeLog][Third-Party Code] Updated CLDR data, used by QLocale, to v45. Task-number: QTBUG-126060 Pick-to: 6.8 6.7 6.5 6.2 Change-Id: I81a5bcca49519b55091fc541de6b73b606661bb4 Reviewed-by: Thiago Macieira <[email protected]>
* Move sorting of likely subtag table upstream to QLocaleXmlReaderEdward Welbourne2024-07-082-22/+36
| | | | | | | | | This means LocaleDataWriter.likelySubtags() now only gets an iterable, so doesn't know when it's on the last item to skip the comma after it, but that seems to be acceptable in modern C++. Change-Id: I9d3bb9af3bb46b28b7a2529e27ab72a72c358503 Reviewed-by: Mate Barany <[email protected]>
* QLocaleXml: unify and shrink language, script and territory listsEdward Welbourne2024-07-082-21/+50
| | | | | | | | | | | | | | | | | The id and code are reliably pure ASCII with no special characters, so can safely be expressed as attributes. Extend the reader and writer classes to handle using attributes on a simple text element. This leaves only the name as text content, so skip the extra <name>...</name> layer. As the resulting element is inside a *List element that tells us whether it's a language, script or territory we don't need to have different elements and can unify them all as simply a <naming id="..." code="...">...</naming> element. This makes these sections of the XML file considerably terser, with no change to the generated data. Change-Id: Id2e884f1d2713341524549cc49253eb33b5aa487 Reviewed-by: Mate Barany <[email protected]>
* QLocaleXml: use tabs for indentationEdward Welbourne2024-06-111-3/+5
| | | | | | | | | | One character instead of four adds up to a lot of saved bytes when a file has many lines: and the timezone name L10n data is going to add a lot of lines. Task-number: QTBUG-115158 Change-Id: I856f3771266a70b7a9ef4078a9b4aecf42315831 Reviewed-by: Mate Barany <[email protected]>
* QLocaleXml: include a <?xml> preambleEdward Welbourne2024-06-111-1/+2
| | | | | | | | Make our encoding explicit and enable more tools to understand what they're looking at. Change-Id: I29327364a5eaac51eeda9a4fb3b8e9b7527ca488 Reviewed-by: Ivan Solovev <[email protected]>
* QLocaleXml: include Qt version in the localeDatabase tagEdward Welbourne2024-06-114-27/+47
| | | | | | | | | | | | | | Also move the CLDR version into the tag. The version numbers are plain ASCII, with no special characters, so can safely be attributes. In the process, fix a mistake in __openTag()'s handling of attributes; join with plain space, no comma. Having the Qt version in the XML makes it possible to assert compatibility between the Qt version that generated it and the one that's consuming it. Change-Id: I6fa6b668b072ff3616955d81af2cffaba5b67250 Reviewed-by: Mate Barany <[email protected]>
* Add --verbose and --quiet arguments to CLDR processing commandsEdward Welbourne2024-06-112-3/+28
| | | | | | | | | | | Support control over verbosity of output. For now just have qlocalexml2cpp.py show a stack-trace when failing (and return on all failures) and have cldr2qlocalexml.py route its information to stdout (when not in use as the XML output stream, else stderr) or discard it in quiet mode. Change-Id: I58afd3a083794eae3a35f6e1235bd62c288fabcf Reviewed-by: Mate Barany <[email protected]>
* Move clearing of self-aliases upstream to QLocaleXmlWriterEdward Welbourne2024-06-052-3/+5
| | | | | | | | | The duplicate entries just bulked up the intermediate file. Makes no change to generated data. Task-number: QTBUG-115158 Change-Id: I6dc0d1f79f8dcf2e46264c6f9d1ae06ff4c91394 Reviewed-by: Mate Barany <[email protected]>
* qlocalexml2cpp.py: rework StringData handling of bit-sizesEdward Welbourne2024-06-021-19/+21
| | | | | | | | | | | Move to construction time, instead of passing to each append() call; the table's field sizes are, after all, the same for all entries. Add support for larger tables by allowing more than 16-bit indices. Task-number: QTBUG-115158 Change-Id: I8f1113482e80838c512da6353fa17b9f365f956a Reviewed-by: Cristian Maureira-Fredes <[email protected]> Reviewed-by: Mate Barany <[email protected]>
* Update C Locale constructor to match others on ids and codesEdward Welbourne2024-06-021-3/+6
| | | | | | | | | | | | | | It was setting *_code='0' for the Any* forms of language, script and territory; this is wrong, the codes for these are all empty or other special tokens (like 'und', 'Zzzz', 'ZZ'). The IDs for them are zero, as an int not a string, but were omitted. Also add the variant details, for all that they're currently unused, for consistency. This makes no difference to the generated data. Task-number: QTBUG-115158 Change-Id: I339d1b201e50e2bbc510758ffbbaae0fa02277d4 Reviewed-by: Mate Barany <[email protected]>
* Derive C locale data from en_US, overriding minor detailsEdward Welbourne2024-06-023-91/+50
| | | | | | | | | | | | | | | | | | | | | | | The qlocalexml.py Locale.C() had to replicate a whole lot of data that isn't really relevant to how C differs from en_US and every addition to what we support required further additions to it. So pass the en_US Locale object to the pseudoconstructor so that C can inherit from it and only override the parts where we care about the difference. Hand-code shortening for short Jalali month names, to match Soroush's original contribution, and include the narrow forms in the hard-coded data to keep the generated data unchanged (for now). Note some of the departures from CLDR; we may want to drop these overrides later. In the process, convert the mapping from keys to locales to consistently use IDs for all members of the key, instead of using the (empty) code value for (as yet unused) variant; it now gets ID 0 and is consistent with returns from codesToIdNames(). This makes life easier for the code that now has to construct an en_US key. Task-number: QTBUG-115158 Change-Id: I3d7acb6a4059daec1bba341fcf015c39c7a6803b Reviewed-by: Kai Köhne <[email protected]>
* qlocalexml2cpp.py: Make clear that ByteArrayData is always ASCIIEdward Welbourne2024-06-021-1/+3
| | | | | | | The container would be unsuitable otherwise. Change-Id: I0b0aa22625fbd638bf8409c5ee257f62332d8e05 Reviewed-by: Mate Barany <[email protected]>
* QLocaleXML: Improve documentation, tidy up a bitEdward Welbourne2024-06-021-15/+25
| | | | | | | | | | | | Omit parentheses round what python will form into a tuple anyway. Include trailing commas on last entries of tuples so adding future entries don't drag the existing line into their diffs. Let the writer's tag-opener handle attributes, if supplied. Clean up spacing in some doc-strings. This is all preparation for further changes, to limit their diffs. Change-Id: I989ae28bbd235b2af9c1d72467d4741c4f1f20ae Reviewed-by: Mate Barany <[email protected]>
* Integrate timezone data into the CLDR-via-QLocaleXml pipelineEdward Welbourne2024-06-027-298/+314
| | | | | | | | | | | | | | | | Future work shall need the timezone alias data to be synchronized between the (expanded) locale-independent timezone data and the (coming) locale-dependent timezone data. The latter shall need to come via QLocaleXml, hence the former now needs to, too. This makes no change to the generated data, aside from changing the regeneration instructions for qtimezoneprivate_data_p.h, to use the same scripts as locale data, instead of cldr2qtimezone.py, which is now removed. Task-number: QTBUG-115158 Change-Id: I47ddd95f6af1855cbb1f601e9074c13f213cd61c Reviewed-by: Mate Barany <[email protected]>
* Add assorted notes and suggestions in util/locale_database/Edward Welbourne2024-06-023-0/+5
| | | | | | Change-Id: I22534943f2c9710d501235672811a861a5fd3aea Reviewed-by: Øystein Heskestad <[email protected]> Reviewed-by: Mårten Nordheim <[email protected]>
* Simplify UTC offset ID data by computing the offsetsEdward Welbourne2024-06-022-50/+46
| | | | | | | | | | It's trivial to do - and done when generating our compiled data tables, so makes no difference to users - but makes the offset list table simpler. Reformat the list so that the fragment-of-hour offsets are clearly distinguished from the whole-hour ones. Change-Id: I6e0ea23dc317542b3256e88492e4073faedef1d7 Reviewed-by: Friedemann Kleint <[email protected]>
* Update the utcIdList (now that I've worked out where it came from)Edward Welbourne2024-06-022-7/+29
| | | | | | | | | | | | | | | | It was originally (without any comment to this effect, either in the code or the commit message) just the list of offset-zones corresponding to known Windows zones' offsets, augmented to include each whole hour offset out to ±14 hours. Absent documentation, of course, this was not maintained. Added the four offset zones implied by that, that hadn't been added when new entries joined the Windows IDs with novel offsets. Check, after scanning CLDR for Windows data, that this has been kept up to date. Updated the generated data. Change-Id: I3cf3932c320876f7f2f74840d8c3951be49cfe70 Reviewed-by: Thiago Macieira <[email protected]>
* Revise Windows time-zone mapping to use proper IANA IDsEdward Welbourne2024-05-302-26/+46
| | | | | | | | | | | | | | | | | | | The CLDR's "IANA" IDs may (for the sake of stability) date back to before IANA's own naming has been updated. As a result, the "IANA" IDs we were using were in some cases out of date. CLDR does provide a mapping from its stable IDs to all aliases and the current IANA name for each (which I shall soon be needing in other work), so use that to map the CLDR IDs to contemporary IANA ones. Revise the documentation of CldrAccess.readWindowsTimeZones() to take this into account, pass it the alias mapping from the table, use that to map IDs internally and, in passing, rename a variable. Update cldr2qtimezone.py to match the new CldrAccess methods and regenerate the data. Change-Id: I23d8a7d048d76392099d125376b544a41faf7eb3 Reviewed-by: Thiago Macieira <[email protected]> Reviewed-by: Mate Barany <[email protected]>
* Use CLDR alias data to find canonical IANA IDsEdward Welbourne2024-05-212-10/+89
| | | | | | | | | | | | | There are various legacy IANA IDs that we should recognize as aliases for their contemporary equivalents. Later work shall also take these into account in the Windows IDs. Scan CLDR's data about these aliases and use it when constructing QTimeZone. This adds aliasMappingTable and aliasIdData arrays to QTZP_data_p.h and an AliasData type to its QtTimeZoneCldr namespace. Change-Id: I1bbfce62959a7e1b7a0bc4a320c32f5a174a2ff2 Reviewed-by: Cristian Maureira-Fredes <[email protected]> Reviewed-by: Thiago Macieira <[email protected]>
* Break out timezone data from cldr2qtimezone.pyEdward Welbourne2024-05-062-213/+235
| | | | | | | | | | | This separates the large slabs of data (and their documentation) from the code that mixes them with CLDR-derived data and generates the data we actually use. In the process, put the shorter table before the longer one, to make it less likely that folk shall fail to notice it's even there at all. Change-Id: I8457741911657dac0dad53c2e65b977821bb4e71 Reviewed-by: Friedemann Kleint <[email protected]>
* Purge an almost-redundant duplicate datetime format conversionEdward Welbourne2024-04-301-60/+4
| | | | | | | | | | | | | | | | | | The QLocale XML reader was passing datetime formats through a format conversion despite the data being converted at the point where we read it from CLDR. It turns out this was needed because the long date and time formats in our hard-coded data for the C Locale object used CLDR format strings, unlike all other Locale objects. Fix those two formats in the C locale and remove the redundant processing step. This, in turn, enables the parser to include the date and time formats in its general handling of most fields that it reads. This does not result in any change to the generated data QLocale uses (although it does change the intermediate QLocale XML file). Task-number: QTBUG-115158 Change-Id: Iaf9da206158043dda2e9e5a3790f009b100e46b4 Reviewed-by: Mate Barany <[email protected]>
* Apply a common style to the main()s of locale database programsEdward Welbourne2024-04-262-9/+30
| | | | | | | | | | | | | | Include documentation in both, using common phrasing. Take sys.argv as a parameter, along with sys.stdout and sys.stderr, so that we can invoke them from python when importing into a python session to debug or test. Supply the script name to the argument parser as prog, so it can correctly report it and forward the rest of argv to parse_args(). Remove comments anticipating one of the several calendars we don't yet support; the existing entries suffice to make clear what shall be needed when we get round to adding more. Change-Id: I2cebd385679e3c84d4ccf899e60091ac823ad10d Reviewed-by: Mate Barany <[email protected]>
* Modernise testlocales/ program and make it compileEdward Welbourne2024-04-264-26/+27
| | | | | | | | | | After several years unused, it had bit-rotted to the point of not compiling and failing an assertion. It also appears to have always had a bad free() on exit, due to passing the address of a static object to a function that took ownership and later deleted it. Change-Id: I91856258c3fedf820bf151b5d205d257876a8e13 Reviewed-by: Jason McDonald <[email protected]>
* Automate updating of list of locales for testlocalesEdward Welbourne2024-04-262-227/+675
| | | | | | | | | | | | | | This old test program has bitrotted due to not being autogenerated as part of CLDR updates. Amend qlocalexml2cpp.py to regenerate it and do such an update. It was still using Qt5's QLocale enum numeric values, many of which have changed in Qt6. Actually fixing the code so that it compiles and runs can wait for a later commit. Inspired by a patch supplied by Kizito Birabwa. Task-number: QTBUG-124200 Change-Id: I33811313976a4860aad6d7b5b88a40c5b111a4fe Reviewed-by: Mate Barany <[email protected]>
* Fix spacing inconsistencies brought to light by flake8Edward Welbourne2024-04-234-5/+5
| | | | | | | | | | It has many grumbles about spacing, but at least this code is currently consistent about its departure from PEP8's spacing rules (and closer to Qt's) for the present. We can review whether to do a drastic spacing revolution later. Change-Id: Ife4e8a5b02b63434bd9c7ac7ba4cbc11b6311f9f Reviewed-by: Mate Barany <[email protected]>
* Fix typo in doc comment for QLocaleXmlWriter.close()Edward Welbourne2024-04-221-1/+1
| | | | | Change-Id: I128ed5e0ebd01a7ed1f3a3049d2b63f1df042562 Reviewed-by: Cristian Maureira-Fredes <[email protected]>
* Use dict comprehensions more in cldr.py and qlocalexml.pyEdward Welbourne2024-04-222-13/+12
| | | | | | | They're a bit more readable than calling dict on a generator. Change-Id: I3177e31b1f617b80d1cf5d5f83df7036fc0c4c01 Reviewed-by: Cristian Maureira-Fredes <[email protected]>
* Tweak the message for variantsEdward Welbourne2024-04-221-1/+5
| | | | | | | | | Although the code does not, in fact, know about them, it's more pertinent to say that they're unsupported than to say that the variant in question is unknown. Change-Id: I411d792dc91f2d7af58a4b7919c952a005b3417e Reviewed-by: Cristian Maureira-Fredes <[email protected]>
* Improve fidelity of approximation to CLDR zone representationsEdward Welbourne2024-04-221-4/+18
| | | | | | | | | | | | | | | | | | | | I neglected to update the CLDR dateconverter code when I expanded the range of forms we support for display of a timezone. Even that expanded range doesn't cover all the cases CLDR does, but we can at least approximate each of CLDR's options by the closest we do support. Make matching changes to how the Darwin backend for the system locale maps its ICU-derived formats to ours. This in practice changes all locales previously using t (abbreviation) as zone format to use tttt (IANA ID) instead. Test data updated to match. [ChangeLog][QtCore][QLocale] Date-time formats now more faithfully follow the CLDR data in handling timezones. In most cases this means the IANA ID is used in place of the abbreviation. Change-Id: I0276843085839ba9a7855a78922cffe285174643 Reviewed-by: Thiago Macieira <[email protected]>
* Correct handling of 'u' in CLDR date format stringsEdward Welbourne2024-04-191-0/+7
| | | | | | | | | | | It explicitly excludes having a two-digit special case like 'yy'. Correct that in qlocale_mac.mm, add support in dateconverter.py No current locale actually uses the 'u' format, so this makes no change to data. Change-Id: I16dfed2d3a7d2054b4b86f9a246bff297df9fc0a Reviewed-by: Dennis Oberst <[email protected]> Reviewed-by: Thiago Macieira <[email protected]>
* Fix handling of am/pm indicators in mapping from CLDR to Qt formatsEdward Welbourne2024-04-191-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | Both qlocale_mac.mm and dateconverter.py were mapping the CLDR am/pm indicator, 'a', to the Qt format token 'AP', forcing the indicator to uppercase. The LDML spec [0] says: May be upper or lowercase depending on the locale and other options. [0] https://www.unicode.org/reports/tr35/tr35-68/tr35-dates.html#Date_Field_Symbol_Table We don't support the "other options" mentioned, but we can at least (since 6.3) preserve the the locale-appropriate case, instead of forcing upper-case. As such, this change is a follow-up to commit 4641ff0f6a1b0da6f55db5e33c58a77be2032808 Changes locale data, as expected, to use "Ap" in place of "AP" in various formats in the time_format_data[] array. [ChangeLog][QtCore][QLocale] Where CLDR specifies an am/pm indicator, the case of the CLDR-supplied indicator is used, where previously QLocale forced it to upper-case. Change-Id: Iee7d55e6f3c78372659668b9798c8e24a1fa8982 Reviewed-by: Konstantin Ritt <[email protected]> Reviewed-by: Thiago Macieira <[email protected]>
* Cope with CLDR's "day period" format specifiersEdward Welbourne2024-04-191-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The LDML spec includes a 'b' pattern character which is like the 'a' pattern, for AM and PM, but would rather use noon and midnight indicators for those specific times. We don't support those and using am/pm will be right enough of the time to be better than simply discarding this option, if it ever gets used (which it currently isn't), so treat as an alias for 'a'. No locale in CLDR currently uses this. CLDR also has a 'B' specifiers for "flexible day periods", including things like "at night" and "in the day". At present only zh_Hant uses 'B'. As a result, this change only affects zh_Hant's formats for time and datetime, which only zh_Hant_TW uses - zh_Hant_HK overrides them to use am/pm markers and zh_Hant_MO inherits that from zh_Hant_HK. Based on this and user feed-back, I've opted to treat 'B' as another synonym of 'a'. This removes an entry from the time_format_data[] table (it happened to occupy one whole twelve-character row), causing many other locales' offsets into that table to be shifted by 12. Only zh_Hant_TW has an actual change to which entry in the table it uses. Added a test-case. [ChangeLog][QtCore][QLocale] CLDR's 'B' (flexible day period, e.g. "at night" &c.) field, not currently supported, is now handled as a synonym for the AM/PM field 'a', instead of leaving the B as literal text. Only affects zh_TW at present. Fixes: QTBUG-123872 Change-Id: I6ba008c0a048190bf7af8c7df7629a885b05804f Reviewed-by: Thiago Macieira <[email protected]>
* Rewrite CLDR-ingestion's date-time format conversionEdward Welbourne2024-04-191-78/+167
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rework the somewhat ad-hoc handling of format blocks. Instead of converting one character at a time, then coming back to map contiguous chunks of various lengths to Qt's best match, use the first non-separator character to select a function that looks ahead to see what to consume with it. Quoted text can be handled the same way, with a look-ahead. This potentially allows for more flexible parsing in future. In the process, matching qlocale_mac.mm, treat all unquoted letters as reserved. The LDML spec says: Currently, A..Z and a..z are reserved for use as pattern characters (unless they are quoted, see next item). and its description of literal text explcitly says these reserved characters are not to be understood as literals. Document the letters we do know about as unsupported pattern characters, but don't do anything specific to handle them. This transiently changes zh_TW's "Bh" hour fields to plain "h" but an imminent commit will change that again and there is no other change to data, so the locale data is not regenerated in this commit, to save churn. This makes the parsing front-end function more straightforward and makes it easier to document the quirks of the different format letters and the impedance mismatches between CLDR's and Qt's. In the process, recognize C, like j and J, as special magic to ignore and harmonize with what qlocale_mac.cpp's macToQtFormat() does, where it's right and dateconverter.py differed. Document the need to stay in sync with this last. Task-number: QTBUG-123872 Change-Id: I490d395b37751c9b8d6f3ee5ed4edbc0d405db5b Reviewed-by: Mate Barany <[email protected]>
* Move LocaleScanner's INHERIT check from find upstream to __findEdward Welbourne2024-04-191-3/+12
| | | | | | | | | | | | | | | | | When digesting CLDR v44.1's github form, some data (e.g. pt_BR's language endonym) were None that had perfectly sensible values in the zip-file form. Letting __find() yield INHERIT entries lead to find() sometimes returning None, where __find() should have tried harder or raised an Error. This further amends commit bcdd51cfae24731a73d008add23d3c1e85bbd8d0 (after commit 0f770b0b34bcb5fa0a598b2ff76fe215fbc25f5c isolated its magic value). Pick-to: 6.7 Task-number: QTBUG-115158 Change-Id: I1af92a687cd50b8fd026c25f068c804a3516ef95 Reviewed-by: Mate Barany <[email protected]>
* Rework enumdata.py's commentsEdward Welbourne2024-03-181-28/+54
| | | | | | | | | Turn the large comment at the start into a doc-string and add some more details to it. Fix the Ivory Coast comment's indent and a typo in it. Change-Id: I36b4e5094d3c3d5c5b91809424b424bcac5daafa Reviewed-by: Friedemann Kleint <[email protected]>
* Minor tidy-up of CldrAccess.__enumMap: revise comment, modernize codeEdward Welbourne2024-02-131-5/+5
| | | | | | | | | | | | A comment dated from when variables misleadingly named language_list, script_list and country_list actually held mappings not lists; they've been renamed to s/list/map/ a while back, so rephrase. Use a dict-comprehension rather than the somewhat lisp-ier invocation of the dict constructor on an iterator over pairs. Change-Id: Ibcb97122434122dbb1dcb0f621aae02b25a4e1fa Reviewed-by: Cristian Maureira-Fredes <[email protected]>
* Move QTimeZone's CLDR-derived data into a namespaceEdward Welbourne2024-02-081-3/+4
| | | | | | | | Introduce namespace QtTimeZoneCldr instead of having a Q prefix on each class name used for the data. Change-Id: Icb22a91340b67f9cc93173b77374a70f69f81bbe Reviewed-by: Ivan Solovev <[email protected]>
* Document LocaleScanner's constructorEdward Welbourne2024-02-081-0/+7
| | | | | | | | I needed to know in order to make recent changes. Save the need to work it out again next time. Change-Id: Ibc606cbe2e6af16e6820fd753a643331a03cdfb3 Reviewed-by: Ievgenii Meshcheriakov <[email protected]>
* Update QLocale and calendar data to CLDR v44.1Edward Welbourne2024-02-021-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (This turns out to be identical to v44, for our purposes.) The CLDR license has been revised at v44 to "UNICODE LICENSE V3", which is now included (as LICENSES/UNICODE-3.0.txt) in addition to the old license (still in use, presumably, by UCD - at least until its next update). Some new QLocale::Language entries are needed. There is no change to the time-zone data. Some tests needed changes: * Various Arabic locales now use U+0623 (Arabic letter aleph with hamza above) in exponent separator, replacing plain U+0627 (Arabic letter aleph); it is still followed by U+0633 (Arabic letter seen). * Where likely sub-tags used to fill in world, 001, as territory for a language, they now (e.g. for Prussian and Yiddish) give specific countries. * Tamil locales now have something of a mix of inherited and localized forms for AM/PM, which looks a lot like a mistake in CLDR. * New likely sub-tag rules fix ctor(und_US) and ctor(und_GB), which previously failed. [ChangeLog][Third-Party Code] Updated QLocale's data extracted from the Unicode Common Locale Data Repository (CLDR) to v44.1. The license changed to Unicode License V3. Pick-to: 6.7 6.6 6.5 Fixes: QTBUG-121485 Task-number: QTBUG-121325 Change-Id: Ide1a68016129526d7a5aa3fc67f1a674858696bc Reviewed-by: Qt CI Bot <[email protected]> Reviewed-by: Mårten Nordheim <[email protected]>
* Fix ordering of Windows timezonesEdward Welbourne2024-02-011-8/+15
| | | | | | | | | | | | | | | | | | | | | The list is meant to be sorted in increasing order, requiring "<anything> (Mexico)" to appear after "<anything>" but in two out of four cases such pairs were in the wrong order. China sorts after Chatham Island and lexical sorting of numbers doesn't match sorting by numeric value. Assert the expected ordering. (The more important check needs a QBAV::compare(), which isn't constexpr, so we can't static_assert.) Later commits shall use binary chop exploiting this ordering. The assertion failed without the rest of this change. Also improve the comments describing the data tables these searches check and the types of their entries. Some were inaccurate, others merely unclear. Likewise, comment the sorting expectations in the python code that generates the tables. Change-Id: I640a3cca8f820c5fd5939a2fe5feb96b04407335 Reviewed-by: Thiago Macieira <[email protected]>
* Move special-case LDML value to a module globalEdward Welbourne2024-01-291-2/+4
| | | | | | | | | | Giving it a symbolic name is clearer (and saves me the need to duplicate the comment when I add some more references to it). This amends commit bcdd51cfae24731a73d008add23d3c1e85bbd8d0 Task-number: QTBUG-115158 Change-Id: I7577e0cde783fcda840009c7aea46934964c6e4c Reviewed-by: Cristian Maureira-Fredes <[email protected]>
* ldml.LocaleScanner.__find(): only Error if no matches foundEdward Welbourne2024-01-291-9/+15
| | | | | | | | | | | | The existing caller returns early on finding a match, so never ran off the end of the iteration unless there were no matches. I'll soon be adding a new caller that wants to iterate all matches, so will run off the end even when there are some. So only raise the Error if we found nothing. Task-number: QTBUG-115158 Change-Id: I1cae4674eb5e83c433554c15ecc4441b756f20eb Reviewed-by: Cristian Maureira-Fredes <[email protected]>
* Package DOM attributes for Node objectsEdward Welbourne2024-01-291-3/+8
| | | | | | | | | | The Supplement type did the needed mapping (using nodeValue when the value wasn't a string) and it turns out to be useful to do the same for the DOM object packaged by a Node, too. Pull out into a helper function, use dict-comprehension and expose as a method of Node. Change-Id: Ice6737a54a33372b45cf42152e3fdbf5f2da7ba4 Reviewed-by: Cristian Maureira-Fredes <[email protected]>
* Prepare to support taking CLDR data from its github upstreamEdward Welbourne2024-01-192-11/+20
| | | | | | | | | | | | | | We've previously used the zip-file form, but that's not been published for CLDR v44.1 - the advice on the list was to use github instead. That, however, has ↑↑↑ as a special value for fields, meaning to inherit from a prent locale. So special-case that value. I have verified that v44 from the zip file produces identical results to v44 from github, with this minor fix. As it happens v44.1 also produces identical results. Pick-to: 6.7 6.5 Change-Id: I6eb0aedda7556753cdc83bb9d76652fbb68dc669 Reviewed-by: Ievgenii Meshcheriakov <[email protected]>
* Convert UTC offset table look-ups to binary chopEdward Welbourne2023-11-031-4/+9
| | | | | | | | | | | | | | | | | | | The table was almost sorted by offset - its UTC entry, with offset 0, was at the front rather than first among the offset 0 entries. The lookups in it were being done as if the IDs were in space-joined lists (as for the IANA IDs in the Windows table), splitting on space, despite the fact that it had separate entries for different IDs at the same offset (this only arose for offset 0). So actually massage the input table in python to combine IDs with the same offset using space, placing UTC first among the offset 0 entries, and ensure the C++ table is sorted. Regenerated the CLDR data tables using the updated script. In the process, fix an off-by-one error in the iteration over space-joined IDs, where the search only advanced to the space, rather than to just after it. That wasn't a problem before, but now would be. Change-Id: Ib49c27bac269b557166fa10738c3e396d58456c0 Reviewed-by: Thiago Macieira <[email protected]>
generated by cgit v1.2.3 (git 2.25.1) at 2025-11-03 09:03:13 +0000