Adds pin.yaml/pin.lock and the vendored diff2html bundle. Templates not
yet updated to reference /static/vendor/, and tailwind still served from
the manually-copied static/tailwind.js (blocked on git-pkgs/pin#2).
Anchors .gitignore vendor/ rule to repo root so it stops matching
internal/server/static/vendor/.
- Implement /julia/* handler for the Pkg server protocol
(registries, registry, package, artifact, meta)
- Resolve package UUIDs to names by parsing Registry.toml from
the General registry tarball, with a hash-guarded background
refresh on registry updates
- Wire into router, ecosystem list, install page, badge styles
- Update README and architecture docs
- Bump github.com/git-pkgs/registries to v0.6.0: the fetcher now
honours HTTP_PROXY, gates dialled IPs against the safehttp block
list, and Version.Integrity is populated for pub, julia and nuget
- Replace internal/cooldown with github.com/git-pkgs/cooldown v0.1.1
(identical surface, lifted from this repo)
- Update docs/architecture.md to point at the external package
Bake the extended linter set into a project config so plain
golangci-lint run matches what we check locally, with goconst tuned
to ignore tests and bare lowercase words to drop ~200 ecosystem-name
and test-literal false positives.
Clear the remaining real findings: extract GradleBuildCacheConfig.Validate
from Config.Validate, pull the eviction sort comparator into
sortOldestFirst (zero time.Time already sorts first via Before so the
switch was redundant), add headerAcceptEncoding and SQL column-type
constants, and drop a dead empty-key recheck in the gradle handler.
* add Gradle Build Cache support with handler and tests
* linting issue
* MR Suggestions: Add Gradle HTTP Build Cache configuration to README
* implement minor stuff: Refactor Gradle handler to remove unnecessary URL parameter and update related tests
Co-authored-by: Copilot <copilot@github.com>
* Add Gradle build cache configuration and eviction support
- Introduced configuration options for Gradle build cache in config files and documentation.
- Implemented read-only mode and upload size limits for the Gradle build cache.
- Added cache eviction logic based on age and size, with corresponding tests.
- Enhanced storage interfaces to support listing objects by prefix.
* implement minor stuff: Refactor Gradle handler to remove unnecessary URL parameter and update related tests
* last finding fix
* fix tests and implement PR suggestions
Co-authored-by: Copilot <copilot@github.com>
* unify path
---------
Co-authored-by: Mateusz (Mati) Kepa <m.kepa@sportradar.com>
Co-authored-by: Copilot <copilot@github.com>
checkCache opened the storage reader and streamed it to the client
without checking that the bytes still matched what was originally
stored, or what the upstream registry declared. Disk corruption,
accidental overwrites, or local tampering would go unnoticed.
Wrap the storage reader in a verifyingReader that computes SHA256
(against artifact.content_hash) and, when version.integrity holds an
SRI string, the corresponding sha256/384/512 digest as bytes flow
through. At EOF the digests are compared; on mismatch we log at
error level, bump proxy_integrity_failures_total, and clear the
artifact's cache entry so the next request refetches from upstream.
Verification is skipped when the stream was not fully consumed
(client disconnect) to avoid evicting good artifacts on partial
reads. The DirectServe presigned-URL path is unverified since the
proxy never sees those bytes.
Refs #42 (part 1)
* Structured JSON error responses for API endpoints
API handlers returned errors via http.Error (text/plain) with ad-hoc
strings, while the mirror API used a different {"error": "..."} shape
and leaked internal err.Error() text to clients.
Add ErrorResponse{Code, Message} with stable codes (BAD_REQUEST,
NOT_FOUND, UPSTREAM_ERROR, INTERNAL_ERROR) and writeError/badRequest/
notFound/internalError helpers. Convert all JSON API handlers in
api.go, browse.go, mirror_api.go and the /stats endpoint. Enrichment
failures now report 502 UPSTREAM_ERROR rather than 500.
Protocol handlers in internal/handler/ are deliberately unchanged
since npm/pip/cargo clients expect their own response formats, not
JSON. HTML page handlers in server.go also keep text/plain.
Swagger @Failure annotations updated and docs regenerated.
Fixes#76
* Convert validatePackagePath errors to JSON in API handlers
The wildcard package routes (/packages/{ecosystem}/*, /api/package/*,
/api/vulns/*, /api/browse/*, /api/compare/*) only checked for an empty
path before passing user input to GetPackageByEcosystemName and the
enrichment service.
Add validatePackagePath as a coarse first-line filter: reject null
bytes, other control characters, and paths over 512 bytes. Wired into
all five entry handlers immediately after the chi wildcard is read.
This is the generic layer; ecosystem-specific name format rules (npm
scoped name shape, Maven coordinate structure, etc.) can be added on
top per #75.
Fixes#75
containsPathTraversal only checked literal ".." segments separated by
forward slashes. Encoded forms like %2e%2e%2f or backslash separators
would slip past if a caller ever passed a raw or Windows-style path.
The check now URL-decodes the input and treats backslashes as
separators before splitting. Go's stdlib already decodes r.URL.Path so
the encoded case is mostly belt-and-braces for cache keys and other
non-router inputs, but the storage layer guard from #106 makes this
worth locking in with tests.
Fixes#74
The browse and compare handlers buffer the full artifact into memory for
prefix detection. Without a cap, a single request for a large cached
artifact could exhaust server memory.
These file types were served with executable content types (text/html,
image/svg+xml) allowing stored XSS via package archive contents.
Also adds Content-Security-Policy: sandbox and X-Content-Type-Options:
nosniff headers to all browse file responses.
Closes#99. The max_size storage config was parsed and validated but
never enforced. This adds a background eviction loop that periodically
checks total cache size and evicts least recently used artifacts when
the limit is exceeded.
When the proxy reaches storage at an internal address (127.0.0.1, a
Docker service name) the presigned URLs it generates point there too,
which is useless to external clients. This adds an optional base URL
that replaces the scheme and host of signed URLs before they're returned,
keeping the signed path and query intact.
When storage.direct_serve is enabled and the backend supports it (S3,
Azure), cached artifact downloads return a 302 redirect to a presigned
URL instead of streaming bytes through the proxy. Falls back to
streaming when the backend can't sign (fileblob, local filesystem) or
signing fails.
Adds the azureblob driver so azblob:// storage URLs work.
Cache-hit accounting already happened before io.Copy so redirects are
counted correctly; the metrics calls are pulled into a helper so both
paths share them.
Closes#96
Cached metadata is now served directly within a configurable TTL window
(default 5m) without contacting upstream, reducing latency and upstream
load. When upstream is unreachable and the cache is past its TTL, stale
content is served with a Warning: 110 header per RFC 7234.
New config: `metadata_ttl` (YAML) / `PROXY_METADATA_TTL` (env).
Set to "0" to always revalidate with upstream.
- ProxyCached now stores upstream Last-Modified in the cache and uses it
(along with ETag) for conditional request handling, returning 304 when
client validators match. Adds Content-Length to cached responses.
- Handlers calling FetchOrCacheMetadata (pypi, composer, pub, nuget) now
check for ErrUpstreamNotFound and return 404 instead of 502, matching
the existing npm and cargo behavior.
- Mirror jobs report live progress via a periodic callback while running,
so API polls return real counts instead of zeroed progress.
- Registry mirroring removed from CLI flags, API acceptance, README, and
docs since every enumerator was a stub returning "not yet implemented".
- Added tests for the conditional metadata path (ETag/If-None-Match,
Last-Modified/If-Modified-Since, 304 responses, header omission).
- Fix race where runJob could overwrite canceled state set by Cancel()
- Fix Debian ecosystem name inconsistency ("deb" -> "debian")
- Stream metadata responses when caching is disabled to avoid buffering
- Add metadata_cache table to initial schema strings for consistency
- Gate mirror API behind mirror_api config flag (disabled by default)
- Fix goconst lint in metadata_cache_test.go
