With proxies sharing a database, a batch can flush after a newer hit is
already written, and overwriting last_accessed_at and updated_at with
its older time could reorder LRU eviction. Both now keep the later of
the stored and incoming times. The count still adds up.
- Track per-caller request and byte counts in a bounded table
- Export proxy_client_requests_total and
proxy_client_response_bytes_total over a closed label set; addresses
are never labels
- Gate the sources card behind ui_request_sources, off by default,
since /ui carries no authentication of its own
- Move access_log.trust_forwarded_for to a top-level key
- Keep remote as the TCP peer and add remote_ip alongside it
Part 6 of 6 splitting #381 up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- 33 panels, no hardcoded data source UID
- Three ecosystem variables, because the label means three different
things across /metrics: the package record, the request path, and
the handler's own name
Part 5 of 6 splitting #381 up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Add downloaded_bytes, downloads and an ecosystems array, served from
the snapshot the gauges and the page already share
- Add stats_unavailable so a failed aggregation is distinguishable
from an idle proxy
- Regenerate the OpenAPI spec
Part 4 of 6 splitting #381 up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Ring of download size by ecosystem, cache figures, per-ecosystem
table, vulnerability overview and a Runtime card
- Add metrics.Gather so the page can render counters that were never
in the database
- Add proxy_response_bytes_total and response-writer byte counting
- Serve a retained snapshot behind a staleness banner when the
aggregation fails, rather than rendering old figures as current
- Extract the security overview into a shared component
Part 3 of 6 splitting #381 up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Add GetEcosystemStats, aggregating packages, versions, artifacts,
cache size, downloads and downloaded bytes per ecosystem
- Publish six proxy_ecosystem_* gauges from it on the existing
one-minute cache-stats tick
- Set and selectively delete rather than Reset, so no scrape lands on
a half-populated vector
- Report artifacts with no package row under "unattributed" rather
than dropping them, so the figures still add up
Part 2 of 6 splitting #381 up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
TestRelayRoutes fails intermittently on CI with "write: connection reset
by peer" at the upstream handler's final Flush, on /hex/packages/demo and
/gem/info/demo.
Those two routes are the only ones in the table that fan out a second
upstream request: with cooldown enabled, the hex and gem handlers fetch
version timestamps concurrently with the artifact. The test server answers
that sidecar request with the same 64 KiB chunked 502, but the handler
returns as soon as it sees a non-200 and closes the body unread, so the
proxy drops the connection while the test server is still writing to it.
Whether the reset lands before or after the Flush is a scheduling race,
which is why it only surfaces on loaded runners.
The flush error says nothing about the response under test, which is
asserted on the downstream side, so stop reporting it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Map /apk to alpine, matching what the handler writes to the database
- Report /helm, /homebrew, /generic and /swift under their own names
- Test every one of the 23 mounts, so the next route added cannot
quietly fall through to "other"
Part 1 of 6 splitting #381 up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every cached download wrote its hit in its own UPDATE. SQLite runs on
one connection, so each download was a write transaction the rest
queued behind, holding a warm npm cache near 1,000 requests a second
with the CPU two thirds idle.
Hits are now counted in memory and written in one transaction every
database.hit_flush_interval, "1s" by default, "0" for a write per hit.
A failed write keeps the hits for the next flush, and Close writes
what is pending. The same test then served about 2,200 requests a
second.
Closes#322.
Eviction deleted an object before checking its record still pointed at
it. If a refetch had moved the record, that object was queued and a
request that read the record earlier could still be opening it. Eviction
now clears first, skips the delete if the record moved, and queues the
object when the delete fails after a clear.
Reclaim queues a failed delete again, which moves it behind the rest, so
objects the backend keeps refusing cannot fill every batch.
Also drop the clearArtifactCache passthrough and ClearCachedArtifact's
unused context, note that only tests call ArtifactPath, and document that
max_size does not count objects waiting to be reclaimed.
Every fetch of an artifact wrote one path, so fetches with different URLs
or digests, which do not share a coalescing key, could overwrite each
other's bytes, serve the wrong ones, or delete them on a digest mismatch
while the other was about to open them.
storeArtifact now writes {ecosystem}/{name}/{version}/{fetch id}/{filename},
with a random id per fetch. Most ecosystems learn the digest only after
the fetch, and storage has no rename, so the id is the one rule that fits
all of them. Existing records keep their paths and stay readable. On a
file:// bucket, Delete now removes an emptied fetch directory, which
fileblob leaves behind, and no other directory, since another fetch may
be creating its own inside it.
A path a record stops pointing at is queued in pending_deletes rather
than deleted, since a request that read the record may still open it. A
loop deletes queued paths after max(1h, direct_serve_ttl), whether or not
max_size is set. UpsertArtifact and clears apply only if the record still
holds the path the caller read, so racing commits each queue the path
they replaced and a clear never orphans a newer commit. Eviction still
deletes inline, and counts space as freed only when the record still
pointed at what it deleted.
