- Keep a gzip copy of each rewrite cache entry when it is smaller, made once
per rewrite and counted against metadata_rewrite_cache_size
- Serve it with Content-Encoding: gzip to clients that accept gzip, with an
ETag of its own ("<sha256>-gzip") and Vary: Accept-Encoding
- Rewrites made outside the cache (cache disabled, cooldown on) stay
uncompressed
- rewriteCache.get and rewrite return rewrittenMetadata
Last-Modified was upstream's date clamped to the process start, so
different documents could share a date: after a restart, a cached document
and the newer one a refresh fetched were both dated at startup, and a client
holding the first got 304 for the second.
- Date each rewrite by when the rewrite cache first made it, and keep that
date while the entry stays cached
- Give every new rewrite of a package a date later than the last one it got,
even within the same second or after its entry was evicted, so no two
representations share a date; dates are remembered only until the clock
passes them
- Send no Last-Modified when the rewrite cache is off, since nothing records
when an uncached rewrite was made; cooldown still sends none
- Drop the upstream Last-Modified from metadata lookups, which nothing uses
now
- With cooldown on every rewrite runs, and hashing each rewritten body for
its ETag made every npm metadata request about 20% more expensive
- Hash what fixes the output instead: the upstream digest, proxy URL,
package name, the versions cooldown and the denylist kept, and a
per-process nonce standing for the startup configuration
- Carry the stored document's digest and upstream Last-Modified with the
metadata lookup result, so neither is read from a second cache row lookup
that a concurrent refresh could have changed
- Fall back to hashing the body when the digest is unknown, and keep the
body hash with cooldown off, where it is computed once per cached rewrite
- Rewritten metadata went out with only Content-Type, so pnpm, npm and
Composer could never revalidate and downloaded every document in full
- ETag is the SHA-256 of the rewritten body, computed once per rewrite and
kept with the rewrite cache entry; exact under cooldown as well
- Last-Modified is the upstream Last-Modified, never earlier than the process
start, since a restart can change the rewrite; omitted with cooldown on
- Answer If-None-Match, and If-Modified-Since when there is no If-None-Match,
with 304 on the rewrite fast path and on the fetch path
The npm and Composer handlers rewrite every metadata document they serve
so that download URLs point at the proxy: decode the whole document into
generic maps, change the URLs, encode it again, and for Composer expand
the minified format first. That ran on every request, cached metadata
included. With metadata caching on and upstream out of the picture, a
cached request still cost 7 ms for @babel/core, 17 ms for
symfony/console and 155 ms and 134 MB of allocations for typescript, and
an 8 vCPU VM serving cached npm packuments to 20 clients ran the proxy
at 450% CPU for about 1,100 requests a second.
Rewritten documents are now kept in memory, keyed by the ecosystem,
proxy URL, package and a SHA-256 of the raw document, so new bytes from
upstream are rewritten again and nothing is served stale. Requests that
arrive while a document is being rewritten wait for that rewrite rather
than running their own; a waiter leaves when its client does, and the
rewrite still completes and is cached. The denylist is fixed at startup,
so it needs no place in the key. Cooldown filtering depends on the
current time, so with cooldown on the cache is bypassed.
metadata_rewrite_cache_size bounds the cache (default "256MB", least
recently used out first, "0" to rewrite on every request). NewProxy
callers keep the old behaviour unless they set it.
Whole cached requests, measured locally:
@babel/core 7.1 ms -> 0.38 ms, 54,489 -> 119 allocations
typescript 155 ms -> 10.5 ms, 1.37M -> 133 allocations
symfony/console 17.4 ms -> 0.48 ms, 165,909 -> 123 allocations
What remains is reading the raw document from storage and hashing it.
