Repository navigation
icu: generate all of ICU's data from text (no prebuilt .dat or property tables) - #836
Draft
bryan-minimal wants to merge 3 commits into
Draft
bryan-minimal wants to merge 3 commits into
bryan-minimal wants to merge 3 commits into
Conversation
The sources tarball ships the data prebuilt (icudt78l.dat). Building from icu4c-78.3-data.zip with the ICU tools built here gives a byte-identical archive and libicudata (all 4305 items). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ICU's release tarballs also ship its core property tables prebuilt (data/in/*.icu, *.nrm, coll/*.icu and the *_props_data.h headers compiled into libicuuc). The release-78.3 tag carries their text inputs and generators; minibazel.py builds gennorm2, genprops and genuca from ICU's BUILD.bazel files with gcc, and the recipe runs them as ICU's data/unidata/generate.sh does. Every installed library is byte-identical to the previous recipe's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
packages/icu(78.3) built fromicu4c-78.3-sources.tgz, which ships ICU's data prebuilt:data/in/icudt78l.dat(33 MB): the whole locale, collation, break-iterator and converter database;data/in/*.icu,*.nrm,coll/*.icu, plus the*_props_data.h,propname_data.handnorm2_nfc_data.hheaders compiled intolibicuuc.makeonly repackaged these intolibicudata/libicuuc. They reach every ICU consumer at runtime,including
node/node-lts(--with-intl=system-icu). A content census of node's whole closure found themto be the only prebuilt binaries that reach its installed output.
This PR builds ICU from the release-78.3 git tag, where all of this exists as text, and generates
every data file in the build:
icu4c/source/data/unidata/generate.shdoes,and fails if a
.datis present.gennorm2,genprops,genuca) from ICU'sBUILD.bazelfiles,without Bazel.
minibazel.pyevaluates those files, which use only plaincc_library/cc_binary, and compiles the targets with the toolchain gcc/g++.generate.shdoes: normalization data fromunidata/norm2/*.txt, properties fromppucd.txt, and collation fromFractionalUCA.txt.genrb/makeconv/genbrk/… (4,073 resource bundles, 190 converters).Evidence (lab, cg-fresh amd64)
generate.shwith Bazel at the tag: all 22 regenerated files are byte-identical to theshipped ones.
minibazel.py): all 22 are byte-identical..datpresent, the builticudt78l.datis byte-identical to the shippedone (all 4,305 items).
libicudata,libicuucandlibicui18nare byte-identical(253/253 installed files). The one exception is
usr/lib/icu/78.3/pkgdata.inc, which records thebuild's CFLAGS, including the source path.
Notes
ppucd.txt, ICU's preparsed form of Unicode's UCD, made bytools/unicode/py/preparseucd.py;regenerating it from the raw UCD is a possible follow-up;
FractionalUCA.txtfrom CLDR;python(no cycle; python does not depend on icu).gs://minimal-staging-archives/icu-release-78.3.tar.gz(sha256
f06bcab7…, read back).unchanged.
🤖 Generated with Claude Code