Pull from icu4c github release instead of package #36

srl295 · 2019-05-14T00:36:33Z

See ICU-20600 for the ICU side. Consider pulling the little endian data directly from the ICU src file (https://github.com/unicode-org/icu/releases/download/release-64-2/icu4c-64_2-src.tgz). BE and EBCDIC could pull from an adjunct data file also published by ICU. This would break the dependency on the manual step of (me) remembering to upload the ICU data.

This also would make sure the dot-release issue mentioned in #35 does not occur, because fetching would be specific to the ICU dot release.

Also, npm install in postinstall is a bad idea, as #38 points out.

The text was updated successfully, but these errors were encountered:

srl295 · 2019-05-15T22:30:28Z

Downside: unable to leverage npm package caching, proxies, etc. However, ICU releases are on github (with its CDN) instead of a single icu-project server these days.
Upside: is no more manually having to publish packages to icu4c-data@latest for each ICU release.
Other Upside: is to no longer have this odd package that installs another package, breaking dependency analysis and the package install model in general.

srl295 · 2019-05-16T18:01:44Z

If it's not a package, this functionality (the fetch) could move into core… into configure.py… into the node installer (check this box to download full ICU)…?

richardlau · 2019-05-17T10:33:02Z

Sounds good. Will this just be for future versions of ICU or will the data files be retroactively published by the ICU project?

srl295 · 2019-05-17T21:19:40Z

@richardlau maybe retroactively publish for current ICU 64.2 — that's going to be what Node LTS v10 uses soon enough. But not all versions. Actually, for little endian non-EBCDIC, the data file is already in the .src.tgz and so several versions back are already available.

caub · 2019-08-30T18:17:09Z

What should we do with this https://github.com/unicode-org/icu/releases/download/release-64-2/icu4c-64_2-src.tgz? should it be saved/uncompressed in a specific directory?

we were using full-icu in https://github.com/brigand/jellobot/blob/master/src/plugins/js-eval/Dockerfile, but recently it stopped working since node v11.8.0 or a little before

srl295 · 2019-08-30T18:29:49Z

https://github.com/unicode-org/icu/releases/download/release-64-2/icu4c-64_2-src.tgz

If you're on little endian, then pull out icu/source/data/in/icudt*l.dat from that zip and copy it to a directory specified in NODE_ICU_DATA (env variable) or --icu-data-dir=… (node parameter) … see https://github.com/nodejs/node/blob/master/doc/api/intl.md#providing-icu-data-at-runtime

if you're on big endian you need to build ICU and convert the data.

recently it stopped working

can you clarify? what stopped working?

caub · 2019-08-30T18:53:24Z

Oh sorry, it's working

docker run --rm node:slim bash -c "npm i full-icu; NODE_ICU_DATA='node_modules/full-icu' node -p 'new Intl.DateTimeFormat(\"fr-FR\", {dateStyle:\"full\"}).format(new Date())'"

vendredi 30 août 2019

thanks, nvm

srl295 · 2019-10-09T21:09:41Z

in light of nodejs/node#29522 I think ICU4C 65.1 is a good cutover time to start this new approach. This means no more updates to the icu4c-data repo.

srl295 · 2020-03-24T19:08:28Z

started on this https://github.com/unicode-org/full-icu-npm/tree/pull-l-from-gh

ex1st · 2020-04-09T05:25:50Z

Hello @srl295

in light of nodejs/node#29522 I think ICU4C 65.1 is a good cutover time to start this new approach. This means no more updates to the icu4c-data repo.

I think, need for at least one time more - #46
Thank you!

ex1st · 2020-04-09T11:24:51Z

@srl295 I've tried npm i unicode-org/full-icu-npm#pull-l-from-gh on v12.16.2 (Ubuntu 19.10), but got:

Error: EACCES: permission denied, mkdtemp '/tmpXXXXXX'
    at Object.mkdtempSync (fs.js:1773:3)
    at ClientRequest.<anonymous> (/home/user/work/node_modules/full-icu/myFetch.js:32:29)
    at Object.onceWrapper (events.js:417:26)
    at ClientRequest.emit (events.js:310:20)
    at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:603:27)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:119:17)
    at TLSSocket.socketOnData (_http_client.js:476:22)
    at TLSSocket.emit (events.js:310:20)
    at addChunk (_stream_readable.js:286:12)
    at readableAddChunk (_stream_readable.js:268:9) {
  errno: -13,
  syscall: 'mkdtemp',
  code: 'EACCES',
  path: '/tmpXXXXXX'
}

srl295 · 2020-04-09T15:38:10Z

@ex1st Sorry, not done yet. I'll get back to work on it!
( I didn't know about npm i … #!

srl295 · 2020-04-10T04:42:52Z

@ilya I’ll have to pick this up Monday. Meanwhile if you go to site.icu-project.org you will find that the icu4c source .zip and .tar contain the icudt*l.dat file under Icu/source/data/in

srl295 · 2020-04-10T04:48:40Z

@ex1st nevermind, i'm uploading it now

srl295 · 2020-04-10T18:56:02Z

please note that ICU 67 RC is the first to 'natively' include the new -l and -b data packages, see https://github.com/unicode-org/icu/releases/tag/release-67-rc (though I want to put the old ones in retroactively)

zen0wu · 2020-07-09T00:43:02Z

Bump bump. Was just bitten by this today. npm install with a cached node_modules changes package-lock.json and invalidates all our node_modules cache. 🙏 🙏

lukeundtrug · 2021-06-15T07:31:19Z

Hi there, just a friendly bump on this promising topic as the current Node updates seem to be missing their respective ICU-data (68l) which led to issues when installing on Node > v14.15.

See #61 for reference 👍

- fetch from ICU’s GitHub release instead of npm (ICU v50+) - set env FULL_ICU_PREFER_NPM to prefer npm instead - add .eslint Fixes: #36

- fetch from ICU’s GitHub release instead of npm (ICU v50+) - set env FULL_ICU_PREFER_NPM=1 to prefer npm instead - add .eslint - for ICU 67 and following, fetch from icu4c-___-data-_.zip otherwise fetch from icu4c-src.zip (will only work for little endian) Fixes: #36

srl295 · 2021-09-24T22:36:55Z

Literally years in the making , PTAL #53

- fetch from ICU’s GitHub release instead of npm (ICU v50+) - set env FULL_ICU_PREFER_NPM=1 to prefer npm instead - add .eslint - for ICU 67 and following, fetch from icu4c-___-data-_.zip - otherwise fetch from icu4c-src.zip for little endian - otherwise, use npm as before Fixes: #36

Fixes: #36 - fetch from ICU’s GitHub release instead of npm (ICU v50+) - set env FULL_ICU_PREFER_NPM=1 to prefer npm instead - add .eslint (as well as standard) - for ICU 67 and following, fetch from icu4c-___-data-_.zip - otherwise fetch from icu4c-src.zip for little endian - otherwise, use npm as before - work around ICU issue with data file name https://unicode-org.atlassian.net/browse/ICU-21764

Fixes: #36 - fetch from ICU’s GitHub release instead of npm (ICU v50+) - set env FULL_ICU_PREFER_NPM=1 to prefer npm instead - add .eslint (as well as standard) - for ICU 67 and following, fetch from icu4c-___-data-_.zip - otherwise fetch from icu4c-src.zip for little endian - otherwise, use npm as before - work around ICU issue with data file name https://unicode-org.atlassian.net/browse/ICU-21764 - document the FULL_ICU_PREFER_NPM - test with FULL_ICU_PREFER_NPM=1

srl295 mentioned this issue May 16, 2019

detect "full-icu" module nodejs/node#3460

Closed

srl295 added the question label May 16, 2019

This was referenced May 30, 2019

Install process is broken/unsafe #38

Closed

OpenJS Summit i18n WG Session Agenda nodejs/i18n#197

Closed

srl295 self-assigned this Aug 30, 2019

srl295 mentioned this issue Nov 13, 2019

tools: Update icu to 65.1 nodejs/node#30232

Closed

4 tasks

Simran-B mentioned this issue Feb 3, 2020

V8: Replace icudtl.dat with full ICU data file 64.2 arangodb/arangodb#11046

Closed

srl295 mentioned this issue Mar 14, 2020

ICU-20600 build icu-data-bin files unicode-org/icu#1048

Merged

srl295 mentioned this issue Apr 10, 2020

Post-Install Failure on node 12.16.2 #46

Closed

srl295 added enhancement and removed question labels Jul 17, 2020

srl295 mentioned this issue Jul 17, 2020

Transfer Unicode ICU modules to Node.js nodejs/admin#532

Closed

srl295 mentioned this issue Aug 3, 2020

Postinstall failes in CI with no public internet and only a private registry #52

Closed

srl295 mentioned this issue May 14, 2021

i18n WG Meeting May 14th nodejs/i18n#493

Closed

srl295 added a commit that referenced this issue Sep 24, 2021

feat: Pull from GitHub instead of npm

c46cc99

- fetch from ICU’s GitHub release instead of npm (ICU v50+) - set env FULL_ICU_PREFER_NPM to prefer npm instead - add .eslint Fixes: #36

srl295 added a commit that referenced this issue Sep 24, 2021

feat: Pull from GitHub instead of npm

dcf0d5b

- fetch from ICU’s GitHub release instead of npm (ICU v50+) - set env FULL_ICU_PREFER_NPM to prefer npm instead - add .eslint Fixes: #36

srl295 mentioned this issue Sep 24, 2021

Pull data files from ICU GitHub instead of npm #53

Merged

srl295 added a commit that referenced this issue Sep 24, 2021

feat: Pull from GitHub instead of npm

e469841

- fetch from ICU’s GitHub release instead of npm (ICU v50+) - set env FULL_ICU_PREFER_NPM to prefer npm instead - add .eslint Fixes: #36

srl295 added a commit that referenced this issue Sep 24, 2021

feat: Pull from GitHub instead of npm

0034ab8

- fetch from ICU’s GitHub release instead of npm (ICU v50+) - set env FULL_ICU_PREFER_NPM to prefer npm instead - add .eslint Fixes: #36

srl295 added this to the v1.4.0 milestone Sep 28, 2021

srl295 closed this as completed in #53 Oct 4, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pull from icu4c github release instead of package #36

Pull from icu4c github release instead of package #36

srl295 commented May 14, 2019 •

edited

Loading

srl295 commented May 15, 2019

srl295 commented May 16, 2019

richardlau commented May 17, 2019

srl295 commented May 17, 2019

caub commented Aug 30, 2019 •

edited

Loading

srl295 commented Aug 30, 2019

caub commented Aug 30, 2019

srl295 commented Oct 9, 2019

srl295 commented Mar 24, 2020

ex1st commented Apr 9, 2020

ex1st commented Apr 9, 2020 •

edited

Loading

srl295 commented Apr 9, 2020

srl295 commented Apr 10, 2020 via email

srl295 commented Apr 10, 2020

srl295 commented Apr 10, 2020 •

edited

Loading

zen0wu commented Jul 9, 2020 •

edited

Loading

lukeundtrug commented Jun 15, 2021

srl295 commented Sep 24, 2021

Pull from icu4c github release instead of package #36

Pull from icu4c github release instead of package #36

Comments

srl295 commented May 14, 2019 • edited Loading

srl295 commented May 15, 2019

srl295 commented May 16, 2019

richardlau commented May 17, 2019

srl295 commented May 17, 2019

caub commented Aug 30, 2019 • edited Loading

srl295 commented Aug 30, 2019

caub commented Aug 30, 2019

srl295 commented Oct 9, 2019

srl295 commented Mar 24, 2020

ex1st commented Apr 9, 2020

ex1st commented Apr 9, 2020 • edited Loading

srl295 commented Apr 9, 2020

srl295 commented Apr 10, 2020 via email

srl295 commented Apr 10, 2020

srl295 commented Apr 10, 2020 • edited Loading

zen0wu commented Jul 9, 2020 • edited Loading

lukeundtrug commented Jun 15, 2021

srl295 commented Sep 24, 2021

srl295 commented May 14, 2019 •

edited

Loading

caub commented Aug 30, 2019 •

edited

Loading

ex1st commented Apr 9, 2020 •

edited

Loading

srl295 commented Apr 10, 2020 •

edited

Loading

zen0wu commented Jul 9, 2020 •

edited

Loading