Commit graph

11 commits

Author SHA1 Message Date
c655399a07 Apply 'go fmt' as suggested in CONTRIBUTING.md. 2026-04-18 07:43:22 -04:00
Andrew Nesbitt
c01f0a5c05
Fix metadata caching, 404 propagation, mirror progress, and registry stubs
- ProxyCached now stores upstream Last-Modified in the cache and uses it
  (along with ETag) for conditional request handling, returning 304 when
  client validators match. Adds Content-Length to cached responses.

- Handlers calling FetchOrCacheMetadata (pypi, composer, pub, nuget) now
  check for ErrUpstreamNotFound and return 404 instead of 502, matching
  the existing npm and cargo behavior.

- Mirror jobs report live progress via a periodic callback while running,
  so API polls return real counts instead of zeroed progress.

- Registry mirroring removed from CLI flags, API acceptance, README, and
  docs since every enumerator was a stub returning "not yet implemented".

- Added tests for the conditional metadata path (ETag/If-None-Match,
  Last-Modified/If-Modified-Since, 304 responses, header omission).
2026-04-13 09:01:05 +01:00
Andrew Nesbitt
d62c42b8d7
Add mirror command and API for selective package mirroring
Add a `proxy mirror` CLI command and `/api/mirror` API endpoints that
pre-populate the cache from various input sources: individual PURLs,
SBOM files (CycloneDX and SPDX), or full registry enumeration.

The mirror reuses the existing handler.Proxy.GetOrFetchArtifact()
pipeline so cached artifacts are identical to those fetched on demand.
A bounded worker pool controls download parallelism.

Metadata caching is opt-in via `cache_metadata: true` in config (or
PROXY_CACHE_METADATA=true). The mirror command always enables it. When
enabled, upstream metadata responses are stored for offline fallback
with ETag-based conditional revalidation.

New internal/mirror package with Source interface, PURLSource,
SBOMSource, RegistrySource, and async JobStore. New metadata_cache
database table for offline metadata serving.
2026-04-13 09:01:04 +01:00
Andrew Nesbitt
e36a92433e
Clean up review feedback: use path.Ext for extension checks, remove dead getStripPrefix, add openArchive tests 2026-04-06 19:06:48 +01:00
Andrew Nesbitt
b68184cbab
Fix composer dist URL rewriting and browse source for extensionless filenames
GitHub zipball URLs end in a bare commit hash with no file extension.
rewriteDistURL now appends .zip when the filename has no extension and
the dist type is zip. expandMinifiedVersions deep copies inherited
values so in-place URL rewriting no longer corrupts shared references.
browse.go infers .zip for extensionless filenames so existing cached
artifacts can still be opened.
2026-04-06 17:07:20 +01:00
Andrew Nesbitt
15c133f1fa
Fix Composer minified metadata expansion and namespaced package routing (#63)
* Fix Composer minified metadata expansion and namespaced package routing

Packagist serves metadata in a minified format where only the first version
entry has all fields and subsequent entries inherit from the previous one.
The proxy was passing this through without expanding it, which meant cooldown
filtering could break the inheritance chain (losing fields like `name`) and
`~dev` sentinel markers were silently dropped.

The proxy now expands the minified format before filtering and rewriting,
ensuring every version entry is self-contained.

Web UI and API routes used single-segment chi URL params for package names,
which broke for Composer's `vendor/name` format. `/package/composer/monolog/monolog`
would match the version show route instead of the package show route.

All `/package/` and related API routes now use wildcard paths with a
`resolvePackageName` helper that tries increasingly longer path prefixes as
package names via DB lookup, correctly handling namespaced packages across
all endpoints (show, version, browse, compare, vulns).

Fixes #61, fixes #62

* Add namespaced package routing tests for all affected ecosystems

Verifies the wildcard routing handles slashes in package names for
npm (@babel/core), Go modules (github.com/stretchr/testify),
OCI images (library/nginx), Conda (conda-forge/numpy), and
Conan (zlib/1.2.13@demo/stable).

* Regenerate swagger docs after route refactor

The swagger annotations for the old per-endpoint handlers were removed
during the wildcard routing refactor. Regenerate to match current state.
2026-04-06 13:07:02 +01:00
Andrew Nesbitt
599fe9e254
Fix all golangci-lint issues across the codebase (#32)
* Fix all golangci-lint issues across the codebase

Resolve 77 lint issues reported by golangci-lint with gocritic, gocognit,
gocyclo, maintidx, dupl, mnd, unparam, ireturn, goconst, and errcheck
enabled. Net reduction of ~175 lines through shared helpers and
deduplication.

* Suppress staticcheck SA1019 for intentional deprecated field usage

The Storage.Path field is deprecated but still read for backwards
compatibility with existing configs that haven't migrated to the URL field.
2026-03-18 10:59:29 +00:00
Andrew Nesbitt
e2a683c7a6
Route handler metadata requests through Proxy.HTTPClient instead of http.DefaultClient
All handler metadata and proxy requests were using http.DefaultClient directly,
bypassing any timeout or transport configuration. Added an HTTPClient field to
the Proxy struct with a 30-second default timeout, and updated every handler
to use it for upstream HTTP requests.
2026-03-13 07:46:28 +00:00
Andrew Nesbitt
0e1a06c5e6
Add size limits on request bodies and upstream metadata reads
POST endpoints (/api/outdated, /api/bulk) now reject bodies over 1 MB
using http.MaxBytesReader. Upstream metadata reads (npm, pypi, composer,
nuget, pub) now use io.LimitReader capped at 50 MB to prevent OOM from
unexpectedly large responses.
2026-03-13 07:28:20 +00:00
Andrew Nesbitt
4f8f63f354
Add version cooldown to filter recently published packages
Hides package versions published too recently from metadata responses,
giving the community time to spot malicious releases. Configurable
per-ecosystem and per-package with duration overrides. Supported for
npm, PyPI, pub.dev, and Composer.
2026-03-04 19:00:31 +00:00
Andrew Nesbitt
7b22638ef7
Hello world 2026-01-20 22:00:31 +00:00