Watch
1
0
Fork
You've already forked pkg-proxy
1
mirror of https://github.com/git-pkgs/proxy.git synced 2026-11-12 00:59:16 -05:00
Commit graph pkg-proxy/internal/handler/npm_rewrite_bench_test.go
Author SHA1 Message Date
Andrew Nesbitt
ac137a3289
Extract portable npm metadata transformations 2026-10-18 17:59:14 +01:00
simon chrzanowski
1584499ac3
Cut memory use of npm metadata rewriting (#413)
Rewriting a packument decoded the whole document into map[string]any and
encoded it again. For a full packument of tens of megabytes that costs many
times the document size per request, and with cooldown enabled (which skips
the rewrite cache and forces full packuments) it happens on every request.

- Rewrite npm metadata in place: walk the raw JSON for member offsets, decode
  only the version names, time and dist-tags for the existing filters, and
  assemble the response from slices of the original bytes with only tarball
  URLs (and filtered time/dist-tags) written fresh. Untouched values now reach
  clients byte for byte instead of round-tripping through encoding/json.
- Look up versions[v].dist.tarball and time[v] directly on the tarball
  download path instead of decoding every version.
- Record the stored metadata's SHA-256 in metadata_cache.content_digest and
  key the rewrite cache by it, so a repeat npm or Composer request inside the
  TTL is served from the rewrite cache without reading or hashing the stored
  document.

On a synthetic 25 MB packument with 5000 versions, rewriteMetadata drops from
~150-174 MB / 706k allocations per call to ~31 MB / 90k, and runs in about
two thirds of the time. npmVersionTarball drops from ~1 MB / 10k allocations
to one allocation.
2026-10-08 07:36:33 +02:00