Commit graph

9 commits

Author SHA1 Message Date
Andrew Nesbitt
7346008aa5
Add metadata TTL and stale-while-revalidate support
Cached metadata is now served directly within a configurable TTL window
(default 5m) without contacting upstream, reducing latency and upstream
load. When upstream is unreachable and the cache is past its TTL, stale
content is served with a Warning: 110 header per RFC 7234.

New config: `metadata_ttl` (YAML) / `PROXY_METADATA_TTL` (env).
Set to "0" to always revalidate with upstream.
2026-04-13 09:01:05 +01:00
Andrew Nesbitt
47681066b5
Fix review issues in mirror feature
- Fix race where runJob could overwrite canceled state set by Cancel()
- Fix Debian ecosystem name inconsistency ("deb" -> "debian")
- Stream metadata responses when caching is disabled to avoid buffering
- Add metadata_cache table to initial schema strings for consistency
- Gate mirror API behind mirror_api config flag (disabled by default)
- Fix goconst lint in metadata_cache_test.go
2026-04-13 09:01:04 +01:00
Andrew Nesbitt
d62c42b8d7
Add mirror command and API for selective package mirroring
Add a `proxy mirror` CLI command and `/api/mirror` API endpoints that
pre-populate the cache from various input sources: individual PURLs,
SBOM files (CycloneDX and SPDX), or full registry enumeration.

The mirror reuses the existing handler.Proxy.GetOrFetchArtifact()
pipeline so cached artifacts are identical to those fetched on demand.
A bounded worker pool controls download parallelism.

Metadata caching is opt-in via `cache_metadata: true` in config (or
PROXY_CACHE_METADATA=true). The mirror command always enables it. When
enabled, upstream metadata responses are stored for offline fallback
with ETag-based conditional revalidation.

New internal/mirror package with Source interface, PURLSource,
SBOMSource, RegistrySource, and async JobStore. New metadata_cache
database table for offline metadata serving.
2026-04-13 09:01:04 +01:00
Andrew Nesbitt
599fe9e254
Fix all golangci-lint issues across the codebase (#32)
* Fix all golangci-lint issues across the codebase

Resolve 77 lint issues reported by golangci-lint with gocritic, gocognit,
gocyclo, maintidx, dupl, mnd, unparam, ireturn, goconst, and errcheck
enabled. Net reduction of ~175 lines through shared helpers and
deduplication.

* Suppress staticcheck SA1019 for intentional deprecated field usage

The Storage.Path field is deprecated but still read for backwards
compatibility with existing configs that haven't migrated to the URL field.
2026-03-18 10:59:29 +00:00
Andrew Nesbitt
4f8f63f354
Add version cooldown to filter recently published packages
Hides package versions published too recently from metadata responses,
giving the community time to spot malicious releases. Configurable
per-ecosystem and per-package with duration overrides. Supported for
npm, PyPI, pub.dev, and Composer.
2026-03-04 19:00:31 +00:00
Andrew Nesbitt
fcc5289f97
Add auth pass-through for upstream registries
Configure authentication per URL prefix in config:

  upstream:
    auth:
      "https://registry.npmjs.org":
        type: bearer
        token: "${NPM_TOKEN}"

Supports bearer tokens, basic auth, and custom headers.
Credentials can reference environment variables with ${VAR_NAME} syntax.
The longest matching URL prefix wins when multiple patterns match.
2026-01-29 16:33:09 +00:00
Andrew Nesbitt
ba754f8a79
Add gocloud.dev/blob for S3 and filesystem storage
Replace custom filesystem storage with gocloud.dev/blob for unified
storage backend support.

Supported backends:
- file:///path/to/dir - Local filesystem (default)
- s3://bucket-name - Amazon S3
- s3://bucket?endpoint=http://localhost:9000 - S3-compatible (MinIO)

Configuration via:
- CLI flag: -storage-url
- Environment: PROXY_STORAGE_URL
- Config file: storage.url

The old storage.path config is deprecated but still supported.
2026-01-29 16:13:16 +00:00
Andrew Nesbitt
41aa11ab66
Add sqlx with SQLite default and PostgreSQL option
Replace raw database/sql with jmoiron/sqlx for cleaner query handling.
Support both SQLite (default) and PostgreSQL as configurable backends.

Configuration via:
- CLI flags: -database-driver, -database-path, -database-url
- Environment: PROXY_DATABASE_DRIVER, PROXY_DATABASE_PATH, PROXY_DATABASE_URL
- Config file: database.driver, database.path, database.url

Tests run against both databases when PROXY_DATABASE_URL is set.
2026-01-29 16:06:56 +00:00
Andrew Nesbitt
7b22638ef7
Hello world 2026-01-20 22:00:31 +00:00