download packages directly from npm

[Rich-Harris]

Alternative to #1244. Rather than fetching files one at a time, we can just reuse the existing mechanism for downloading tarballs. This way, we download way fewer files, with fewer waterfalls and fewer opportunities for network flakiness.

In some cases it does mean downloading more bytes, because a) you might only be using one of several exposed entrypoints, and b) package authors can be lazy about what goes into their packages. But anecdotally the playground seems faster with this approach (though it'll be interesting to take a look at a preview deployment).

We do still need to use a package CDN, since the npm registry API doesn't seem to offer a way to resolve incomplete versions (e.g. you can't do @1 or @1.2, you have to do @1.2.3), and the 'packument' (yes they really call it that) that contains details of all the published versions can be a very large file. So we fetch a package.json from a package CDN as a relatively cheap way to determine the exact version. This PR uses jsDelivr rather than unpkg.

WIP:

• need to populate imports for the sake of downloads
• need to get version=local working (which I suspect will mean creating an endpoint that serves the local files as a tarball)
• deduplicate code between bundler (in packages/repl) and compiler (in packages/editor)

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
svelte-dev	✅ Ready (Inspect)	Visit Preview	Apr 3, 2025 9:55pm

[Rich-Harris]

Three is an example of an unusually large package — an extreme example of 'fewer downloads but more bytes'. To me it's noticeably quicker with this PR:

• today
• this PR

[dummdidumm]

This looks promising! My biggest fear is that for some reason Cloudflare in front of npm starts to think we're a bot and blocks us from downloading tarballs (sometimes when I go through many npm packages in quick succession I get a Cloudflare challenge).

Tried to find some random REPLs on the internet and the following failed:

• https://svelte-dev-git-npm-svelte.vercel.app/playground/915db3b3ed704fddb7ddfb64bcbc2624?version=5.25.6
• https://svelte-dev-git-npm-svelte.vercel.app/playground/4bc37ff40ada4111b71fe292a4eb90f6?version=5.25.6
• https://svelte-dev-git-npm-svelte.vercel.app/playground/hello-world?version=5.25.6#H4sIAAAAAAAACq1SwW7bMAz9FUKXblgStwG2gxMP29ING9BDgfa27KDYtC1MpgyJdpoY-fdBSlzHQW7rSdDj46PE9zpBskIRi3sri3tr6gflGL6_yKrWKCYiVxqdiH93gne153lATPqur3U9cy1q9thGOryGp4YYiZ2IxdKlVtX8eU1rVlVtLEP3KAtFko09QG5NBTdfslxLi9FRYFrLAqckW1VIVoZuFmvy7RoZfMlBAvPbxZoAPJQ21iLxoywQEvi0gCiCXwypJNggmEoxYwYqh51pYCuJgQ00DoFLhAxz2WiGVuoGweRwC72uIsWrsqG_T2rvle-Gkb70VJrtD2Udf2uYDUECbBv8z-leAi6nPMi3H9Lv05VmuxoW-CA3qN9mSi61w6NxeUOpNzK4tyolFfhTUqbRvsMWid9D52lrHlsZarMMWSo9860-B3zwmstoiBUtM9WGfJ0-Ar4_hu5c7QPcHXzbiUrL1wyC7wyxSrpw-AHnL0nOhUJxlIykG11fCRf5ONIuwBF5sHngDligXnMr6a6hgW4oTsO2jz8bbf4A0Xh5z3YHFp3aKyqCo1uVcRnyEi6Ume1og453GkNrptreQcYXnkqtCoohRWK0i2MhN8RTp_YYw3z20WJ1witpC0UxzC1WIBs2vcnLqB8gJsLLitin8vBnInwg_INEHEJ2-AcQ8fjj0wQAAA== (copied from https://svelte.dev/playground/9aaf93575c8f45c2ae5525f6f1dc3874?version=5.25.6 which for some reason isn't available on the preview)

[dominikg]

checked the ones above:

• https://svelte-dev-git-npm-svelte.vercel.app/playground/915db3b3ed704fddb7ddfb64bcbc2624?version=5.25.6

has a peerDep on svelte3 and uses the old syntax of a .svelte file in package.json svelte field (and main field for good measure)
https://www.npmjs.com/package/svelte-dragdroplist?activeTab=code

---

• https://svelte-dev-git-npm-svelte.vercel.app/playground/4bc37ff40ada4111b71fe292a4eb90f6?version=5.25.6

tinro also uses a svelte field in package.json to expose its es.js file.
https://www.npmjs.com/package/tinro?activeTab=code

---

• https://svelte-dev-git-npm-svelte.vercel.app/playground/hello-world?version=5.25.6#H4sIAAAAAAAACq1SwW7bMAz9FUKXblgStwG2gxMP29ING9BDgfa27KDYtC1MpgyJdpoY-fdBSlzHQW7rSdDj46PE9zpBskIRi3sri3tr6gflGL6_yKrWKCYiVxqdiH93gne153lATPqur3U9cy1q9thGOryGp4YYiZ2IxdKlVtX8eU1rVlVtLEP3KAtFko09QG5NBTdfslxLi9FRYFrLAqckW1VIVoZuFmvy7RoZfMlBAvPbxZoAPJQ21iLxoywQEvi0gCiCXwypJNggmEoxYwYqh51pYCuJgQ00DoFLhAxz2WiGVuoGweRwC72uIsWrsqG_T2rvle-Gkb70VJrtD2Udf2uYDUECbBv8z-leAi6nPMi3H9Lv05VmuxoW-CA3qN9mSi61w6NxeUOpNzK4tyolFfhTUqbRvsMWid9D52lrHlsZarMMWSo9860-B3zwmstoiBUtM9WGfJ0-Ar4_hu5c7QPcHXzbiUrL1wyC7wyxSrpw-AHnL0nOhUJxlIykG11fCRf5ONIuwBF5sHngDligXnMr6a6hgW4oTsO2jz8bbf4A0Xh5z3YHFp3aKyqCo1uVcRnyEi6Ume1og453GkNrptreQcYXnkqtCoohRWK0i2MhN8RTp_YYw3z20WJ1witpC0UxzC1WIBs2vcnLqB8gJsLLitin8vBnInwg_INEHEJ2-AcQ8fjj0wQAAA== (copied from https://svelte.dev/playground/9aaf93575c8f45c2ae5525f6f1dc3874?version=5.25.6 which for some reason isn't available on the preview)

this one is made with svelte-package, has exports condition and type: module so its a bit more tricky. Maybe it has to do with a .svelte file in that package importing another .svelte file?

[Rich-Harris]

@MartinKolarik thanks for the tip! made that change

[Rich-Harris]

Okay I got very carried away here — the code was long overdue for a bit of an overhaul, and this makes some progress in that direction (merging packages/repl and packages/editor, replacing the $bundle store with modern state, etc). There's more I want to do but I'll stop here for now, I think it's ready for review

[benmccann]

Three is an example of an unusually large package — an extreme example of 'fewer downloads but more bytes'. To me it's noticeably quicker with this PR:

today
this PR

I tried to test this on mobile where I'd be most worried about extra bytes and could never get the unpkg one to finish loading. The version for this PR was also quite good - admittedly on 5g, which is probably quite a bit better than 3g. But this PR was a clear winner for me.

@PuruVJ you might want to give this a spin both as someone who has the least internet bandwidth among us and as someone who's worked on this in the past

[PuruVJ]

I can confirm that this is nearly as fast for me. On multiple hard reloads over VPN, I can't say the new version is faster, but since it does not feel slower either over VPN, I can say this is the right direction to go forward in

[Rich-Harris]

Another very stark example is Lucide:

• today — straight-up fails most of the time, if it does succeed then it's pretty slow and fetches like 1,700 files
• this PR — honey badger don't care

[Rich-Harris]

(currently trying to diagnose an absolutely insane bug — was randomly trying out stuff from my /apps list and this freezes for reasons i absolutely cannot fathom)