Watch
1
0
Fork
You've already forked pkg-proxy
1
mirror of https://github.com/git-pkgs/proxy.git synced 2026-11-12 17:19:17 -05:00
Commit graph pkg-proxy/internal/handler/conda.go
Author SHA1 Message Date
Giles Westwood
b2955ef458 Keep holds through late successes and away from the circuit breaker
A response arriving during a hold was sent before it began, so a late
success no longer releases the host; only one after the hold has ended
does.

Artifact fetches check RateLimitGate.Held before calling the fetcher, so
held cache misses fail with the HeldError without counting as failures
in the host's circuit breaker.

Every handler that answers an upstream fetch error with 502 now answers
a held request with 503 and Retry-After instead.

OnHold and RecordUpstreamRateLimitHold drop the unused duration.
2026-10-15 13:35:12 +01:00
Christian Heim
c58cdaece9 Refuse conda rules with more than a channel and a name
Downloads are routed as /{channel}/{arch}/{filename}, so the stored name is always {channel}/{name}. A key like pkg:conda/conda-forge/label/rc/numpy passed validation and matched nothing; it's now refused.

Refs #306
2026-10-12 19:59:35 +02:00
Christian Heim
bffba28262 Enable storage retention for conda
The handler stores packages as {channel}/{name}, which purl writes as pkg:conda/conda-forge%2Fnumpy. Package rules name the channel as the namespace (pkg:conda/conda-forge/numpy) and get mapped to that form, with the package name lowercased the way conda treats it. A key without a channel or with an empty part could never match and is refused.

The mirror can't fetch conda packages today, so there's no mirror test.

Refs #306
2026-10-12 19:56:45 +02:00
PhantomPhoton
f60772d2d8
Support HTTP byte-range requests for local file cached artifact downloads (#404)
* feat(storage): expose efficient seek for local files

Open file-backed artifacts with os.Open so local readers support
efficient seeking. Leave cloud storage readers unchanged.

Keep the legacy sidecar mapping test focused on fileblob's reader:
Blob.Open now reads local files directly and no longer consults the
sidecar. Continue checking sidecar cleanup and post-store reads.

* feat(handler): preserve efficient seek through integrity checks

Keep the cache result's single Reader field and preserve io.Seeker
when the underlying storage reader supports efficient seeking.

Sequential reads continue through whole-object integrity verification.
A successful seek switches subsequent reads to the source for range
responses, where full-object verification is not possible.

Test both modes and verify the source is closed once.

* feat(handler): add request-aware byte range responses

Add a request-aware artifact serving helper for bounded, open-ended,
and suffix byte ranges, including If-Range handling, 206 responses,
and 416 responses for valid unsatisfiable ranges.

Ignore malformed and multi-range requests by serving the full artifact.
Advertise byte-range support only for seekable readers, preserving HEAD,
redirect, and non-seekable behavior. Clarify that net/http recovers
ErrAbortHandler per request and keeps the server running.

* feat(handler): enable byte ranges across artifact downloads

Forward each artifact request to the shared range-aware response helper,
including cached generic release assets and OCI blobs. This lets the
handler honor ranges only when its reader supports efficient seeking,
without changing direct-storage redirect behavior.

Add warm-cache endpoint tests that assert partial response headers and
bytes without an upstream refetch. Document supported range behavior,
unsupported reader fallback, and the integrity limitation of partial reads.
2026-10-04 17:48:41 +01:00
abhinavgautam01
ac261cc34d
fix: centralize pass-through response relaying 2026-09-26 15:44:28 +05:30
abhinavgautam01
e40c78f72a
fix(conda): preserve gzip for large repodata indexes 2026-09-25 21:12:00 +05:30
Andrew Nesbitt
c5e14835dd
Use header constants across the handler package (#301)
goconst tripped on main after #259 landed on top of #280: five
composite-literal occurrences each of "Content-Length" and
"Content-Type" across container.go, container_manifest.go,
container_tags.go, handler.go, and swift.go crossed the
min-occurrences: 5 threshold. Neither PR hit it alone.

Add headerContentType and headerContentLength beside
headerAcceptEncoding and use them throughout the package rather than
only at the flagged sites, so the next handler that adds one does not
re-trip the check.
2026-09-02 13:22:02 +01:00
Andrew Nesbitt
f43aa9d13e
Make built-in upstream URLs configurable (#255)
Every built-in ecosystem upstream can now be set via the upstream
config block or PROXY_UPSTREAM_* env vars, with allow_private_hosts
and allow_loopback controlling access to non-public addresses.
2026-08-29 11:19:54 +01:00
Tilian Honig
532e4925fe
fix: proper handling of upstream registry 404s (#209)
* fix: proper handling of upstream registry 404s

* fix: consistently return 404s for all artifact types
2026-07-26 19:07:46 +01:00
Andrew Nesbitt
cdbd1a9369 Canonicalize cooldown lookup PURLs to match config
NormalizedPackages runs config keys through purl.Parse().String(), which
applies per-type rules like lowercasing pypi names. The handler side was
building lookup keys with MakePURLString, which does not, so a config
entry for pkg:pypi/Django would be normalized to pkg:pypi/django and
never match the runtime key pkg:pypi/Django.

Route both sides through the same canonical form: a new
canonicalPackagePURL helper calls Normalize() on the constructed PURL
before stringifying, and all cooldown IsAllowed call sites use it.
2026-07-17 18:11:47 -07:00
Andrew Nesbitt
0e7af4aed6
Make metadata size limit configurable (closes #149) (#150) 2026-06-02 07:59:00 +01:00
Andrew Nesbitt
992f5c68a7
Add .golangci.yml and clear gocognit/goconst findings (#113)
Bake the extended linter set into a project config so plain
golangci-lint run matches what we check locally, with goconst tuned
to ignore tests and bare lowercase words to drop ~200 ecosystem-name
and test-literal false positives.

Clear the remaining real findings: extract GradleBuildCacheConfig.Validate
from Config.Validate, pull the eviction sort comparator into
sortOldestFirst (zero time.Time already sorts first via Before so the
switch was redundant), add headerAcceptEncoding and SQL column-type
constants, and drop a dead empty-key recheck in the gradle handler.
2026-05-05 10:25:17 +01:00
Andrew Nesbitt
d62c42b8d7
Add mirror command and API for selective package mirroring
Add a `proxy mirror` CLI command and `/api/mirror` API endpoints that
pre-populate the cache from various input sources: individual PURLs,
SBOM files (CycloneDX and SPDX), or full registry enumeration.

The mirror reuses the existing handler.Proxy.GetOrFetchArtifact()
pipeline so cached artifacts are identical to those fetched on demand.
A bounded worker pool controls download parallelism.

Metadata caching is opt-in via `cache_metadata: true` in config (or
PROXY_CACHE_METADATA=true). The mirror command always enables it. When
enabled, upstream metadata responses are stored for offline fallback
with ETag-based conditional revalidation.

New internal/mirror package with Source interface, PURLSource,
SBOMSource, RegistrySource, and async JobStore. New metadata_cache
database table for offline metadata serving.
2026-04-13 09:01:04 +01:00
Andrew Nesbitt
75ff85f2f0
Add cooldown support for Conda (#68)
* Add cooldown support for Conda

Filter entries from Conda repodata.json based on the timestamp field
(milliseconds since epoch). Filters both packages and packages.conda
sections. When cooldown is disabled, repodata requests are proxied
directly without parsing.

* Update README table to mark Conda cooldown support
2026-04-06 13:16:00 +01:00
Andrew Nesbitt
599fe9e254
Fix all golangci-lint issues across the codebase (#32)
* Fix all golangci-lint issues across the codebase

Resolve 77 lint issues reported by golangci-lint with gocritic, gocognit,
gocyclo, maintidx, dupl, mnd, unparam, ireturn, goconst, and errcheck
enabled. Net reduction of ~175 lines through shared helpers and
deduplication.

* Suppress staticcheck SA1019 for intentional deprecated field usage

The Storage.Path field is deprecated but still read for backwards
compatibility with existing configs that haven't migrated to the URL field.
2026-03-18 10:59:29 +00:00
Andrew Nesbitt
e2a683c7a6
Route handler metadata requests through Proxy.HTTPClient instead of http.DefaultClient
All handler metadata and proxy requests were using http.DefaultClient directly,
bypassing any timeout or transport configuration. Added an HTTPClient field to
the Proxy struct with a 30-second default timeout, and updated every handler
to use it for upstream HTTP requests.
2026-03-13 07:46:28 +00:00
Andrew Nesbitt
7b22638ef7
Hello world 2026-01-20 22:00:31 +00:00