Compare commits

..
Author SHA1 Message Date
trinity-1686a
8bbf7e98c9 add internals for supporting transactional updates 2026-02-07 13:26:56 +01:00
Gwen Lg
c7d6eb631f chore: update cmd parameter name in shell.nix
should fix the warning :
WARN[0000] Flag --customPlatform is deprecated. Use: --custom-platform
2026-01-29 22:14:34 +00:00
Gwen Lg
5eb381a22a refactor: use OnceLock for Instance in test
... instead of unsafe `static mut` as it's not safe.
Change terminate to apply on `&self` and use a Mutex for Instance interior mutability.
2026-01-29 20:50:05 +01:00
Gwen Lg
3b6daa7d0f refactor: rework init_tracing to avoid unused variable
... when feature `telemetry-otlp` is not enabled
move feature management into tracing_setup module to make the server code cleaner.
2026-01-29 20:50:05 +01:00
Gwen Lg
7d05d9d520 chore: set aws_s3 BehaviorVersion to latest in tests common client
ok because this code shouldn't be reliant on extremely specific behavior characteristics.
And this avoid use of deprecated version.
2026-01-29 20:50:05 +01:00
Gwen Lg
f0b443652a ci: add lints check with clippy in debug workflow 2026-01-29 20:50:04 +01:00
Gwen Lg
cd5cd37ecc style: improve lisibility of db_path code
split suffix get from match self and path construction
2026-01-29 20:49:28 +01:00
Gwen Lg
e5f87fb51e refactor: use let Some instead of is_some + unwrap
lint message: called `unwrap` on `config.k2v_api` after checking its variant with `is_some`
lint message: called `unwrap` on `config.admin.trace_sink` after checking its variant with `is_some`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.93.0/index.html#unnecessary_unwrap
2026-01-29 20:49:28 +01:00
Gwen Lg
b0ee7dd3c9 chore: localy disable some lints
- clippy::nonminimal_bool disabled for check_size_filter function
clippy message: this boolean expression can be simplified
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#nonminimal_bool
- clippy::large_enum_variant for `DecryptStreamState` and `State`
- clippy::too_many_arguments for `put_block_and_meta` and
  `test_read_encrypted`
- clippy::deref_addrof for specific unsafe code
- clippy::doc_overindented_list_items and clippy::doc_lazy_continuation
2026-01-29 20:49:28 +01:00
Gwen Lg
4a0be692b3 docs: various fixes in Rust code documentation
- add backticks on struct doc comments
lint message: unclosed HTML tag `M`
- add a blanck line for proper doc formating
lint message: doc list item without indentation
help: if this is supposed to be its own paragraph, add a blank line
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#doc_lazy_continuation
- fix hyperlink in doc + one url invalid
lint message: this URL is not a hyperlink
note: bare URLs are not automatically turned into clickable links
- ajust space in doc
lint message: doc list item overindented
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#doc_overindented_list_items
2026-01-29 20:49:28 +01:00
Gwen Lg
a5d047b5ae style: use if else instead of then_some + unwrap_or
create dedicated fn for `deleted` printing to improve lisibility an
deduplicate code.

clippy message: this method chain can be written more clearly with `if .. else ..`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#obfuscated_if_else
2026-01-29 20:49:28 +01:00
Gwen Lg
24e11e99ee style: some small fixes
- remove invalid struct pattern
lint message: struct pattern is not needed for a unit variant
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#unneeded_struct_pattern
- remove call to default() on unit struct
lint message: use of `default` to create a unit struct
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#default_constructed_unit_structs
- replace if/else by direct affectation
lint message: this if-then-else expression assigns a bool literal
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_bool_assign
- replace assert_eq on unit by `unwrap` + let typed binding
2026-01-29 20:49:28 +01:00
Gwen Lg
bddd23136b refactor: reduce number of arguments by use struct
- create and use HandleInfo and DestInfo to reduce number of arguments
lint message: warning: this function has too many arguments (9/7)
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#too_many_arguments
2026-01-29 20:49:28 +01:00
Gwen Lg
a426b00e87 style: collapse nested if block
lint message: this `if` statement can be collapsed
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#collapsible_if
2026-01-29 20:49:28 +01:00
Gwen Lg
8772db8228 refactor: rename method to avoid confusion
- rename SqliteDb::new to `_::open` as it's not return Self
lint message: methods called `new` usually return `Self`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#new_ret_no_self
- rename method `add` to `add_calgorithm`
The semantics of this has nothing to do with an `add` operation in the sense of the `Add` trait.
And method `add` can be confused for the standard trait method std::ops::Add::add
lint https://rust-lang.github.io/rust-clippy/rust-1.93.0/index.html#should_implement_trait
2026-01-29 20:49:28 +01:00
Gwen Lg
5307b3f762 chore: remove unnecessary cast usize
lint message: casting to the same type is unnecessary (`u64` -> `u64`)
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#unnecessary_cast

+ disable `clippy::unnecessary_cast` for update_disk_usage
where the size of the input values depends on the platform
2026-01-29 20:49:28 +01:00
Gwen Lg
6dbcdcd784 style: improve use of std api
- replace get(0) by first()
lint message: accessing first element with `objs.get(0)`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#get_first
- use io::Error::other method available since Rust 1.87
lint message: this can be `std::io::Error::other(_)`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#io_other_error
- implement From<_> instead of Into<_>
lint message: an implementation of `From` is preferred since it gives you `Into<_>` for free where the reverse isn't true
help: `impl From<Local> for Foreign` is allowed by the orphan rules, for more information see
            https://doc.rust-lang.org/reference/items/implementations.html#trait-implementation-coherence
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#from_over_into
- use next_back instead of rev + next
lint message: manual backwards iteration
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#manual_next_back
- use saturating_sub instead of manual implement it
lint message: manual arithmetic check found
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#implicit_saturating_sub
- add Default implementation for Checksummer
use derived Default ans use it in new, as new set all value to None
lint message: you should consider adding a `Default` implementation for `Checksummer`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#new_without_default
- use `any` instead of `find` + `is_some`
lint message: called `is_some()` after searching an `Iterator` with `find`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#search_is_some
- replace `and_then` + `Some(_)` with `map`
lint message: using `Option.and_then(|x| Some(y))`, which is more succinctly expressed as `map(|x| y)`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#bind_instead_of_map
- replace `skip_while` + `next` with `find`
lint message: called `skip_while(<p>).next()` on an `Iterator`
help: this is more succinctly expressed by calling `.find(!<p>)` instead
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#skip_while_next
- use `matches!` instead of manual implementation
lint message: match expression looks like `matches!` macro
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#match_like_matches_macro
- use `keys` and `values methods insteads of iterating and ignoring either the keys or values.
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#iter_kv_map
2026-01-29 20:49:28 +01:00
Gwen Lg
f2f6669bf0 style: clean use for question_mark
- remove useless `Ok` enclosing with `?`
lint message: enclosing `Ok` and `?` operator are unneeded
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_question_mark
- use question mark instead of manual implementation
lint message: this `match` expression can be replaced with `?`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#question_mark
2026-01-29 20:49:28 +01:00
Gwen Lg
21db7a4d4b chore: remove various useless code
- call expect directly on Result
lint message: called `ok().expect()` on a `Result` value
help: you can call `expect()` directly on the `Result`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#ok_expect
- use assign operation instead of manual implementation
lint message: manual implementation of an assign operation
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#assign_op_pattern
- remove useless call to format
lint message: useless use of `format!`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#useless_format
- remove useless `?Sized`
lint message: `?Sized` bound is ignored because of a `Sized` requirement
note: ...because `Deserialize` has the bound `Sized`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_maybe_sized
- use is_some instead of pattern maching with Some(_)
lint message: redundant pattern matching, consider using `is_some()`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#redundant_pattern_matching
- remove unneeded unit return type
lint message: unneeded unit return type
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#unused_unit
- remove redundant closure
lint message: redundant closure
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#redundant_closure
- use derive Default instead of manual implementation
lint message: this `impl` can be derived
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#derivable_impls
- remove unneeded `return` statement
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_return
- remove empty string from println call
lint message: empty string literal in `println!`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#println_empty_string
- remove clone() on type than implement Copy
lint message: using `clone` on type `Option<ChecksumValue>` which implements the `Copy` trait
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#clone_on_copy
- remove useless let binding
lint message: this let-binding has unit value
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#let_unit_value
- remove useless len comparison to zero, already test of empty
lint message: length comparison to zero
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#len_zero
- remove useless `as_deref` call
lint message: derefed type is same as origin
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_option_as_deref
- remove useless conversion of the same type
lint message: useless conversion to the same type: `replication_mode::ReplicationFactor`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#useless_conversion
- remove useless bool_comparison
lint message: equality checks against false can be replaced by a negation
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#bool_comparison
- remove useless to_string
lint message: unnecessary use of `to_string`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#unnecessary_to_owned

Signed-off-by: Gwen Lg <me@gwenlg.fr>
2026-01-29 20:49:28 +01:00
Gwen Lg
ea9597819c refactor: clean perf related
- use array instead of vec when it's useless
clippy lint message: useless use of `vec!` help: you can use an array directly.
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#useless_vec
- call as_bytes before slicing
lint message: calling `as_bytes` after slicing a string
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#sliced_string_as_bytes

Signed-off-by: Gwen Lg <me@gwenlg.fr>
2026-01-29 20:49:28 +01:00
Gwen Lg
141b3f24f1 chore: clean relative to references and borrows
- lint message: the borrowed expression implements the required traits
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_borrows_for_generic_args
- lint message: this expression creates a reference which is immediately dereferenced by the compiler
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_borrow
- lint message: you don't need to add `&` to all patterns
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#match_ref_pat
- remove useless taken reference
lint message: needlessly taken reference of left operand
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#op_ref
- use &Path instead of &PathBuf as fn parameters
lint message: writing `&PathBuf` instead of `&Path` involves a new object where a slice will do
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#ptr_arg
2026-01-29 20:49:28 +01:00
Gwen Lg
209263eb93 style: adjust specified lifetime in code
- include the lifetime instead of hide it
lint message: hiding a lifetime that's elided elsewhere is confusing
help: the same lifetime is referred to in inconsistent ways, making the signature confusing
- remove useless lifetime
lint message: the following explicit lifetimes could be elided: 'a
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_lifetimes
2026-01-29 20:49:28 +01:00
Gwen Lg
295c850380 chore: add taplo.toml and resave Cargo.toml
Signed-off-by: Gwen Lg <me@gwenlg.fr>
2026-01-29 20:49:28 +01:00
Gwen Lg
33d37b24bd fix: remove duplicate net feature of tokio
Signed-off-by: Gwen Lg <me@gwenlg.fr>
2026-01-29 20:49:28 +01:00
Gwen Lg
46f6967934 chore: update arc-swap to v1.1 as 1.0 is yanked 2026-01-29 20:49:28 +01:00
Gwen Lg
c0b574361e ci: add typos step in debug workflow
use typos package than can be not up-to-date.
add `typos` in the list of packages of the shell used for all CI jobs
2026-01-29 14:53:27 +01:00
Gwen Lg
f2e00781bb chore: add exceptions in typos conf file
- `PN` use in some tex file
- `substituters` which is an offcial word of Nix vocabulary
2026-01-29 14:53:27 +01:00
Gwen Lg
32da94cbbe chore: rename strat into strategy to improve redability
`strat` is reported as error by typos
2026-01-29 14:53:27 +01:00
Gwen Lg
014bebfa1f chore: improve code readability by rename pn var in part_number 2026-01-29 14:53:27 +01:00
Gwen Lg
e9fbde3adf chore: fix typos in variant Abandoned in enum PeerConnState 2026-01-29 14:53:27 +01:00
Gwen Lg
1d1cfb0e29 chore: fix typos in various files
yml, json, tex, sh
2026-01-29 14:53:27 +01:00
Gwen Lg
e331f88c85 chore: fix typos in rust code comment or error message 2026-01-29 14:53:27 +01:00
Gwen Lg
4650fbd49c chore: fix typos of rust type name in markdown
=> `CustomResourceDefinition` and `metadata_fsync`
2026-01-27 21:17:15 +01:00
Gwen Lg
43ed68c558 chore: a large number of typo corrections in markdown files 2026-01-27 21:17:15 +01:00
Gwen Lg
d1bc921ec2 chore: add 'typos.toml' configuration file 2026-01-27 21:17:15 +01:00
Thijs Broersen
ef36e4c8b2
fix: helm configmap quoted block_size 2026-01-25 13:15:20 +01:00
Alex Auvolat
582b168b6a bump version to v2.2.0 2026-01-24 12:32:22 +01:00
Alex Auvolat
c821d4974a Update cargo dependencies (#1295)
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1295
Co-authored-by: Alex Auvolat <lx@deuxfleurs.fr>
Co-committed-by: Alex Auvolat <lx@deuxfleurs.fr>
2026-01-24 11:23:15 +00:00
Alex
eb1b621a5a Merge pull request 'Sync main-v2 with main-v1' (#1296) from merge-v1 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1296
2026-01-24 10:37:44 +00:00
Alex Auvolat
0c32294485 pin crane version in flake 2026-01-24 10:36:19 +01:00
Alex Auvolat
21500545bb fix python/boto3 2026-01-24 10:12:04 +01:00
Alex Auvolat
a525d0e36a small doc updates 2026-01-24 09:56:22 +01:00
Alex Auvolat
29e869ac88 reenable S3 checksumming in test scripts 2026-01-24 09:56:14 +01:00
Alex Auvolat
cd641a9ed2 Merge branch 'main-v1' into merge-v1 2026-01-24 09:56:02 +01:00
Dryusdan
68876f0b08 Add Pleroma documentation (#1294)
I have successfully setup my Pleroma instance with GarageHQ and would share this in the documentation.

The part with the loop is totally optionnal, but, as I said, in case of old link in Pleroma database I prefer run it.

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1294
Co-authored-by: Dryusdan <contact@dryusdan.fr>
Co-committed-by: Dryusdan <contact@dryusdan.fr>
2026-01-24 08:25:32 +00:00
Alex
ba1f30d393 Merge pull request 'Document feature flags in the cookbook and in --version' (#1286) from jasonaowen/garage:document-feature-flags into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1286
Reviewed-by: trinity-1686a <trinity-1686a@noreply.localhost>
2026-01-20 10:48:08 +00:00
Alex
9018ed9b97 Merge pull request 'Fixes #1268 Arch Linux package in Extra now' (#1284) from kuba86/garage:arch-package-extra-1268 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1284
Reviewed-by: trinity-1686a <trinity-1686a@noreply.localhost>
2026-01-20 10:47:48 +00:00
kuba86
13ded6dd35 Fixes #1268
Update Arch Linux installation instructions in binary-packages guide
2026-01-20 11:45:00 +01:00
Alex
4d96719d96 Merge pull request 'Fix presigned post when bucket is absent from fields' (#1290) from joeanderson/garage:fix/presigned-post-bucket into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1290
Reviewed-by: trinity-1686a <trinity-1686a@noreply.localhost>
2026-01-20 10:39:46 +00:00
joeanderson
77ef3c85ea Merge branch 'main-v2' into fix/presigned-post-bucket 2026-01-18 15:38:28 +00:00
Joe Anderson
79c7126b67
Allow bucket to be missing from presigned post params 2026-01-18 15:08:59 +00:00
Jason Owen
77b6233496
Document remaining features in cookbook & version
Add the remaining feature flags to the cookbook and the version
information.
2026-01-13 22:42:43 -08:00
Jason Owen
547fe30a05
Sort feature flags
Put the features flags into alphabetical order in both the cookbook and
the version information, so that it is easier to document additional
feature flags.
2026-01-13 22:21:38 -08:00
Jason Owen
bff5068efc
Refer to git.deuxfleurs.fr as Forgejo not Gitea
The version control site was migrated from Gitea to Forgejo some time
ago, and Forgejo has declared a hard fork from Gitea[1]. Update the
documentation and links to refer to the site as a Forgejo instance
instead of a Gitea instance.

[1] https://forgejo.org/2024-02-forking-forward/
2026-01-13 20:35:56 -08:00
Alex
60eee993b4 Merge pull request 'Fix typo in error message' (#1283) from rmoff/garage:main-v2 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1283
2026-01-12 21:17:46 +00:00
rmoff
d30bb2acb1 Fix typo in error message 2026-01-12 17:32:02 +00:00
maximilien
730c613807 Merge pull request 'notes about synology hyperbackup' (#1281) from lowcarbdev/garage:hyperbackup-docs into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1281
2026-01-08 18:34:57 +00:00
maximilien
9e356347c6 hyperbackup docs: minor spelling fixes 2026-01-08 18:34:32 +00:00
lowcarbdev
1d3c0511b1 notes about synology hyperbackup 2026-01-07 22:50:45 -07:00
Alex
f50b342c00 Merge pull request 'Adding consul discovery for WAN federated consul servers' (#1252) from jamin/garage:feature/consul_wan_discovery_v2 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1252
2026-01-07 08:22:05 +00:00
Alex Auvolat
cf22e7b71d reintroduce warning when invalid node id is present in consul 2026-01-06 14:35:51 +01:00
Alex Auvolat
dc8d93698b small documentation fixes and simplify config struct 2026-01-06 14:33:20 +01:00
Alex
006e78ccea Merge pull request 'Added a loop to go through bootstrap_peers and make their output TOML friendly, instead of the Go default.' (#1238) from magsol/garage:main-v2 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1238
2026-01-06 09:36:18 +00:00
Alex
276e55ae8b Merge pull request 'helm: Trim extra newline when common labels not set' (#1242) from boris.m/garage:fix/helm-common-labels-templating into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1242
2026-01-06 09:25:44 +00:00
Alex
ab6d9633ac Merge pull request 'openapi: work around issue for flattenned untagged enum (fix #1249)' (#1278) from fix-openapi-enum into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1278
2026-01-06 09:21:20 +00:00
Alex Auvolat
cf2f058f60 openapi: work around issue for flattenned untagged enum (fix #1249) 2026-01-06 10:11:12 +01:00
maximilien
02677af546 Merge pull request 'Adding ente documentation' (#826) from tcheronneau/garage:main into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/826
2025-12-22 01:23:22 +00:00
maximilien
fced78c283
doc: ente doc fixes from comments 2025-12-22 02:18:19 +01:00
maximilien
d211c1e291
ente: add more details on bucket configuration
add more configuration with details for options

Signed-off-by: maximilien <maximilien@deuxfleurs.fr>
2025-12-22 02:18:18 +01:00
Thomas
318ef40c4e
Adding ente documentation 2025-12-22 02:18:15 +01:00
maximilien
c9c50b741e Merge pull request 'cookbook: update arch install repo' (#1272) from update-arch-doc into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1272
2025-12-21 21:15:38 +00:00
nmstoker
a0a2c1db88
Update doc/book/cookbook/binary-packages.md
Correct the Arch Linux link as garage is now available in the official repos under extra, and no longer in AUR.
2025-12-21 22:14:25 +01:00
maximilien
04d549acc7 Merge pull request 'Add conda-forge to binary packages' (#1270) from Cosmos2020/garage:cosmos2020-patch-1 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1270
2025-12-21 21:11:46 +00:00
maximilien
a9cd3f3426 Merge pull request 'feat: add service annotations to main-v2' (#1271) from add-helm-svc-annotation into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1271
2025-12-21 21:11:22 +00:00
Pierre Mavro
8f7e92b6a7
feat: add service annotations 2025-12-21 22:02:36 +01:00
Cosmos2020
c2d54b4136 doc/book/cookbook/binary-packages.md aktualisiert 2025-12-21 17:13:16 +00:00
JaminMartin
2f7a649870 doc: added documentation for WAN service discovery 2025-12-19 11:26:24 +13:00
JaminMartin
c4836916b0 feat: testing wan consul discovery 2025-12-19 10:24:11 +13:00
Alex
b830bdd1dd Merge pull request 'Document workaround for Cloudflare proxy signature error when using rclone' (#1256) from kuba86/garage:document-workaround-for-cloudflare-proxy-when-using-rclone into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1256
2025-12-17 10:03:50 +00:00
maximilien
161b185464 Merge pull request 'docs: fix typo in doc/book/cookbook/kubernetes.md' (#1263) from fix-typo into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1263
2025-12-17 07:47:24 +00:00
maximilien
8f0d10b0b1 docs: fix typo in doc/book/cookbook/kubernetes.md 2025-12-17 08:19:20 +01:00
maximilien
6bac369ee2 Merge pull request 'cookbook/reverse-proxy: remove Buypass Go SSL' (#1258) from ElementW/garage:main-v2 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1258
2025-12-15 09:54:28 +00:00
Céleste Wouters
96d303b05e
cookbook/reverse-proxy: remove Buypass Go SSL
Buypass Go SSL has stopped providing SSL/TLS certificates, including
from their ACME API endpoints, as of October 2025, and the service is to
be completely phased out by April 2026:
https://community.buypass.com/t/y4y130p

Remove them from the docs as an ACME-capable provider.
2025-12-15 02:03:24 +01:00
kuba86
3a1dce59f7 Document workaround for Cloudflare proxy signature error when using rclone 2025-12-13 17:19:32 +01:00
B Marinov
6fd2cf7966 Trim space when common labels is empty 2025-12-05 21:25:02 +01:00
Shannon Quinn
6aecd9718f Added a loop to go through bootstrap_peers and make their output TOML friendly, instead of the Go default. 2025-11-25 15:33:47 -05:00
Alex
0412013229 Merge pull request 'Change optimization level to 3' (#1235) from majudev/garage:main-v2 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1235
2025-11-25 09:57:56 +00:00
majudev
090dbb412a Change optimization level to 3 2025-11-24 18:12:01 +01:00
Alex
12367d307b Merge pull request 'Add additionalTopLevelConfig to helm chart' (#1184) from florissmit10/garage:feature/add-aditional-config-helm into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1184
Reviewed-by: trinity-1686a <trinity-1686a@noreply.localhost>
2025-11-24 09:56:04 +00:00
Alex
675c1c156d Merge pull request 'helm: add existingRpcSecret option' (#1225) from rkevin/garage:krong-helm-add-existingrpcsecret into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1225
2025-11-24 09:55:37 +00:00
Alex
eac3a60050 Merge pull request 'migrate to this error, garage-v2' (#1219) from thiserror-2 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1219
Reviewed-by: Alex <lx@deuxfleurs.fr>
2025-11-12 08:05:43 +00:00
Alex
a5580c99fe Merge pull request 'don't append index document to redirect urls' (#1223) from redirect-no-index into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1223
2025-11-11 17:27:39 +00:00
Alex
8083bb4a0f Merge pull request 'upgrade crc-fast' (#1217) from update-crc-fast into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1217
2025-11-11 17:25:31 +00:00
Alex
00cdefa6b3 Merge pull request 'JOSY talk' (#1191) from talk/josy into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1191
Reviewed-by: trinity-1686a <trinity-1686a@noreply.localhost>
2025-11-11 17:25:01 +00:00
rkevin
4dee3e6e04 helm: add existingRpcSecret option 2025-11-10 20:30:41 -08:00
trinity-1686a
5333285b50 don't append index document to redirect urls 2025-11-06 18:03:58 +01:00
trinity-1686a
ef913843f7 fmt 2025-11-01 19:35:11 +01:00
trinity-1686a
1fe932d07f migrate to this error
it doesn't generate a bazillion warning at compile time
2025-11-01 17:25:23 +01:00
trinity-1686a
7e5bb51287 upgrade crc-fast 2025-11-01 15:43:08 +00:00
Quentin
00a5c3d8a2 Merge branch 'main-v2' into talk/josy 2025-10-28 16:02:29 +00:00
maximilien
2e7a6fccba Merge pull request 'Use OnRootMismatch for fsGroupChangePolicy to avoid prolonged Pod upstart' (#1190) from larssb/garage:on-root-mismatch-pod-upstart-time into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1190
2025-10-07 22:22:55 +00:00
Quentin Dufour
09041035d5
slides 2025-10-07 10:28:27 +02:00
Quentin Dufour
ae64ecf10c
tmp presentation 2025-10-06 22:06:22 +02:00
larssb
c8ac4a2105 Use OnRootMismatch for fsGroupChangePolicy to avoid prolonged Pod upstart
When not used. Especially with a lot of small files on the PVC's Garage uses/is configured with,
Pod upstart time can get quite extreme. For more context see a very good description
of the issues and the technical intricacies behind it here: https://github.com/longhorn/longhorn/issues/2131#issuecomment-778897129

I think it's sane to have fsGroupChangePolicy at "OnRootMismatch" as Garage is an S3 system and thereby
is likely to process a lot of files. And somewhat likely that many of these will be small files as well.

---

Using `fsGroupChangePolicy: "OnRootMismatch"` in my env. changed the Garage Pod upstart. E.g. in an
upgrade case from 5-7minutes ( around 50GB of data ). Many small files though. To 1-15sec.

Thank you.

Signed-off-by: larssb <larssb@noreply.localhost>
2025-10-06 19:45:44 +00:00
Quentin Dufour
280c1303fa
finale structure 2025-10-06 15:48:46 +02:00
Quentin Dufour
0f5b3878ca
WIP new garage presentation 2025-10-06 12:39:43 +02:00
Floris Smit
d863247f9f Add additionalTopLevelConfig to helm chart 2025-10-01 11:34:46 +02:00
Alex
6c740ff05c Merge pull request 'Documentation fixes' (#1169) from fix-doc into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1169
2025-09-15 18:14:34 +00:00
Alex Auvolat
95d9905524 updated instructions about metadata engines and snapshots 2025-09-15 20:12:00 +02:00
Alex Auvolat
53fe77860b update fjall support version info 2025-09-15 19:51:41 +02:00
Alex Auvolat
6d5e971974 fix typo in documentation link 2025-09-15 19:51:41 +02:00
Alex
4d8407dc0f Merge pull request 'garage v2.1.0' (#1168) from rel-v2.1.0 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1168
2025-09-15 16:53:35 +00:00
Alex Auvolat
006fb18aea fix compilation warning 2025-09-15 15:52:53 +02:00
Alex Auvolat
b43f309ec7 bump version to v2.1.0 2025-09-15 15:50:02 +02:00
Alex
df4721387c Merge pull request 'Merge changes from v1.x into v2.x' (#1161) from sync-v2-to-v1 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1161
2025-09-15 13:24:20 +00:00
Alex Auvolat
9c067c0cbd Merge branch 'main-v1' into sync-v2-to-v1 2025-09-14 21:04:04 +02:00
Alex
742129f4a3 Merge pull request 'openapi: fix query parameters (fix #1160)' (#1162) from fix-1160 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1162
2025-09-14 16:46:43 +00:00
Alex Auvolat
7a256b2ebb openapi: fix query parameters (fix #1160) 2025-09-14 17:27:37 +02:00
Alex Auvolat
909359ca4c Merge branch 'main-v1' into sync-v2-to-v1 2025-09-14 17:02:34 +02:00
Alex
3148fa3afe Merge pull request 'add error case for layout not ready, and fail earlier in many places' (#1159) from fix-1107 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1159
2025-09-14 14:45:00 +00:00
Alex Auvolat
6b06459b99 fullcopy tables: fix read quorum 2025-09-13 21:34:09 +02:00
Alex Auvolat
4c139bcbca add error case for layout not ready, and fail earlier in many places 2025-09-13 20:52:51 +02:00
Alex
4758d8881f Merge pull request 'woodpecker: require the nix=enabled label' (#1153) from woodpecker-nix-flag-v2 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1153
2025-09-04 09:15:17 +00:00
Alex Auvolat
61e19310c8 woodpecker: require the nix=enabled label 2025-09-04 11:11:03 +02:00
Alex
17fe11fa81 Merge pull request 'fix(openapi): use query params, not path params for Update & Delete Bucket' (#1139) from milas/garage:milas/fix-openapi-v2-params into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1139
2025-09-04 07:23:45 +00:00
Milas Bowman
16128fca63
chore: re-generate OpenAPI spec
```
cargo run -- admin-api-schema > doc/api/garage-admin-v2.json
```
2025-09-03 21:08:54 -04:00
Alex
29570f3192 Merge pull request 'helm: add commonLabels to all resources and pod templates' (#1135) from Schwitzd/garage:feat/helm-commonLabels into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1135
Reviewed-by: maximilien <me@mricher.fr>
2025-09-02 08:28:35 +00:00
Schwitzd
c35c1b5b9b helm: add commonLabels to all resources and pod templates 2025-09-01 15:51:05 +00:00
Alex
7e203f634e Merge pull request 'fix server panic caused by garage json-api Metrics (fix #1132)' (#1144) from fix-1132 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1144
2025-08-27 22:24:29 +00:00
Alex
99f7c0fc4b Merge pull request 'fix panic in garage status (fix #1131)' (#1145) from fix-1131 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1145
2025-08-27 22:15:15 +00:00
Alex Auvolat
fb95a8819f fix server panic caused by garage json-api Metrics (fix #1132) 2025-08-27 23:59:41 +02:00
Alex Auvolat
665addc03b fix panic in garage status (fix #1131) 2025-08-27 23:58:48 +02:00
Alex
7949927291 Merge pull request 'take consistency_mode (degraded/dangerous) into account in fullcopy tables' (#1143) from fix-fullcopy-quorums into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1143
2025-08-27 21:54:50 +00:00
Alex Auvolat
2ddb29ca35 take consistency_mode (degraded/dangerous) into account in fullcopy tables 2025-08-27 23:34:06 +02:00
Alex
30d8ec5368 Merge pull request 'fix(script/telemetry): remove broken panel override in grafana prometheus dashboard' (#1075) from RockWolf/garage:rockwolf-patch-1 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1075
2025-08-27 16:21:39 +00:00
Alex
47772eb525 Merge pull request 'fix: return consistent cors headers on api error' (#1115) from Xstoudi/garage:fix/consistent-cors into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1115
2025-08-27 16:19:36 +00:00
Alex
c1ed770e64 Merge pull request 'log access keys Garage v2' (#1124) from 1686a/log-access-key-v2 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1124
2025-08-27 16:17:39 +00:00
Alex
7e80e86934 Merge pull request 'Add dev scripts dependencies to devShellFull' (#1134) from atnnn/garage:dev-script-deps into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1134
2025-08-27 16:15:30 +00:00
Milas Bowman
4deb57815a
fix(api): use query params, not path params for Update & Delete Bucket 2025-08-23 13:02:03 -04:00
Etienne Laurin
df343dd808 Add dev scripts dependencies to devShellFull 2025-08-12 19:59:30 -07:00
Alex
17c73bafa2 Merge pull request 'fix: method name duplicated after client sdk generation' (#1111) from MagicRR/garage:fix-openapi-duplicate-method into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1111
2025-08-07 14:32:12 +00:00
MagicRR
d8058e7475 fix: method name duplicated after client sdk generation 2025-08-07 10:14:56 +00:00
trinity-1686a
385fbc606d log user-defined admin api tokens 2025-08-03 15:50:36 +02:00
trinity-1686a
6f9d6919a9 log access keys 2025-08-03 15:31:51 +02:00
Alex
91fde4105d Merge pull request 'hide keys without any permissions from admin api' (#1123) from 1686a/hide-empty-keys into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1123
2025-08-03 11:18:51 +00:00
trinity-1686a
d975960be3 hide keys without any permissions from admin api 2025-08-02 17:28:14 +02:00
Alex
6508acbe71 Merge pull request 'ignore checksums with empty strings' (#1116) from CobaltCause/garage:push-qsqlmltnouyv into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1116
2025-08-02 08:29:10 +00:00
Xavier Stouder
985ad68ade fix: run cargo format 2025-07-31 22:52:27 +02:00
Charles Hall
b7a853b01f ignore checksums with empty/whitespace-only bodies
aws-sdk-cpp was observed to send request bodies like this via Lix:

```xml
<?xml version="1.0"?>
<CompleteMultipartUpload xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Part>
        <ETag>"8a5031bda169553d6a232e6c11068774"</ETag>
        <ChecksumCRC32></ChecksumCRC32>
        <PartNumber>1</PartNumber>
    </Part>
    <Part>
        <ETag>"da977cc58b75bd17749c1ff460ba301a"</ETag>
        <ChecksumCRC32></ChecksumCRC32>
        <PartNumber>2</PartNumber>
    </Part>
</CompleteMultipartUpload>
```
2025-07-31 11:15:33 -07:00
Charles Hall
66faef9fb6 factor out repetitive else-if chain into macro 2025-07-30 18:44:51 -07:00
Charles Hall
13f67b6cd8 log incorrect multipart completion body 2025-07-30 17:08:43 -07:00
Xavier Stouder
0dabf9b22f fix: return consistent cors headers on api error 2025-07-30 22:20:48 +02:00
Alex
e226fb413f Merge pull request 'fix: return 204 on successful AbortMultipartUpload' (#1095) from nikeee/garage:fix-abort-multipart-upload into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1095
2025-07-11 17:12:32 +00:00
Niklas Mollenhauer
708a84f1d6
fix: return 204 on successful AbortMultipartUpload
Docs state that 204 should be returned on success:
https://docs.aws.amazon.com/AmazonS3/latest/API/API_AbortMultipartUpload.html
```http
HTTP/1.1 204
x-amz-request-charged: RequestCharged
```
2025-07-11 16:19:51 +02:00
Alex
0465475599 Merge pull request 'documentation fixes' (#1101) from doc-fixes into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1101
2025-07-11 11:32:01 +00:00
Alex Auvolat
0a45317b3b doc: fix link to k2v spec 2025-07-11 13:30:03 +02:00
Alex Auvolat
bb3b832024 doc: fix scrub tranquility command 2025-07-11 13:27:17 +02:00
Alex
f8be15c37d Merge pull request 'api: add instrospect endpoint' (#1092) from Xstoudi/garage:feature/introspect-endpoint into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1092
2025-07-11 11:20:12 +00:00
Alex
1e05fc1d53 Merge pull request 'fix: GetBucketCORS returns 404 if there are no policies' (#1096) from nikeee/garage:fix-get-bucket-cors into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1096
2025-07-10 09:56:54 +00:00
Alex
e5eff872f5 Merge pull request 'doc: Update use case' (#1098) from jonah/garage:main-v2 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1098
2025-07-10 09:54:47 +00:00
Jonah Aragon
605ee4cdb1
doc: Update use case 2025-07-08 17:46:02 -05:00
Niklas Mollenhauer
71aef8770e
fix: GetBucketCORS returns 404 if there are no policies
Similar issue @ ceph:
https://github.com/ceph/ceph/pull/27122/files
https://tracker.ceph.com/issues/38886

Implementation @ minio:
de234b888c/cmd/dummy-handlers.go (L196)
2025-07-08 17:33:51 +02:00
Xavier Stouder
b4f6ab963c api: correct according to review 2025-07-04 21:36:34 +02:00
Xavier Stouder
9a31b9c077 api: change endpoint name and allow it to be called even if not in current token scope 2025-07-03 21:59:23 +02:00
Xavier Stouder
58a96dc687 api: correct openapi def 2025-07-03 21:59:22 +02:00
Xavier Stouder
7bbb3ff9cf api: add instrospect endpoint
Fixes #1091
2025-07-01 23:14:09 +02:00
Alex
f04af18193 Merge pull request 'reverse-proxy.md: use $http_host instead of $host for alternative port' (#1085) from dongdigua/garage:main-v2 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1085
2025-06-23 07:53:44 +00:00
dongdigua
67e0fcc6ea reverse-proxy.md: use $http_host instead of $host for alternative port
if the nginx port is not 443, the host header won't match, causing a Forbidden: Invalid signature
2025-06-23 01:26:03 +00:00
Alex
78f03aec78 Merge pull request 'fix(openapi): set parameters in query instead of path for get params' (#1082) from Xstoudi/garage:fix/openapi-get-parameter into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1082
2025-06-20 10:25:51 +00:00
Xavier Stouder
56a23d936e chore: regenerate openapi spec v2 2025-06-19 23:55:00 +02:00
Xavier Stouder
9b6e45ca1f fix(openapi): set parameters in query instead of path for get params
get request params are wrongfully outputted as "in path" instead of "in query", it fix this.

Fixes #1081
2025-06-19 22:31:44 +02:00
RockWolf
27666ed265 fix(script/telemetry): remove broken panel override in grafana prometheus dashboard
Signed-off-by: RockWolf <rockwolf@noreply.localhost>
2025-06-16 20:36:15 +00:00
Alex
e8e722cc66 Merge pull request 'update documentation pages on admin API' (#1074) from doc-admin-api into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1074
2025-06-15 13:44:05 +00:00
Alex Auvolat
80f818eb6c update documentation pages on admin API 2025-06-15 15:40:38 +02:00
Alex
f899e023a0 Merge pull request 'documentation fixes' (#1072) from doc-fixes into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1072
2025-06-15 09:28:13 +00:00
Alex Auvolat
7556c536ae documentation fixes 2025-06-15 11:27:21 +02:00
Alex
2a20319fa9 Merge pull request 'openapi spec: add missing datatypes for path parameters' (#1071) from fix-openapi-params into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1071
2025-06-14 15:37:26 +00:00
Alex Auvolat
42baa29e50 openapi spec: add missing datatypes for path parameters 2025-06-14 17:25:37 +02:00
Alex
f461348790 Merge pull request 'migration guide for Garage v2' (#1070) from upgrade-guide-v2 into main-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1070
2025-06-14 15:19:39 +00:00
Alex Auvolat
4a8f7e15ce migration guide for Garage v2 2025-06-14 17:19:04 +02:00
Alex Auvolat
44587d295a Merge branch 'main' into next-v2 2025-06-14 16:03:21 +02:00
Alex Auvolat
dc1a4ffd76 Merge branch 'main' into next-v2 2025-06-13 14:01:39 +02:00
Alex Auvolat
53005c91a5 reduce log level of table sync messages 2025-05-25 18:54:53 +02:00
Alex Auvolat
b7a153b892 CI: fix use of aws cp to talk to older Garage versions
chicken and egg problem
2025-05-23 19:06:58 +02:00
Alex Auvolat
bc8e6af223 update admin API documentation 2025-05-23 17:25:17 +02:00
Alex Auvolat
78b1481461 update version numbers in helm chart 2025-05-23 16:43:14 +02:00
Alex Auvolat
7ab1d176d4 Merge branch 'main' into next-v2 2025-05-23 16:33:07 +02:00
Alex
b15d55ea9f Merge pull request 'implement x-amz-checksum-type' (#1025) from checksum-type into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1025
2025-05-22 17:06:04 +00:00
Alex Auvolat
c13af97b81 return ChecksumType in CompleteMultipartUpload result 2025-05-22 18:24:29 +02:00
Alex Auvolat
d1d5c67ba7 fix test-smoke for CompleteMultipartUpload 2025-05-22 18:14:53 +02:00
Alex Auvolat
77125e9464 add boto3 test for STREAMING-UNSIGNED-PAYLOAD-TRAILER 2025-05-22 17:47:12 +02:00
Alex Auvolat
cfd10480ee CopyObject: fix checksumming choice logic 2025-05-22 17:07:10 +02:00
Alex Auvolat
fbb40c4ea0 object_table: merge checksum_algorithm and checksum_type for Uploading state 2025-05-22 17:07:10 +02:00
Alex Auvolat
e475c7f802 api: switch to crc_fast for checksumming and implement full-object mpu checksums 2025-05-22 17:03:54 +02:00
Alex Auvolat
589a992af8 api: parse x-amz-checksum-type and validate combination with algo 2025-05-22 17:03:54 +02:00
Alex Auvolat
768794daae api: change ExpectedChecksums to not be a reference 2025-05-22 17:03:54 +02:00
Alex Auvolat
abe0546ab0 model: store x-amz-checksum-type (full_object | composite) 2025-05-22 17:03:54 +02:00
Alex
47fe96279b Merge pull request 'throw error on legacy replication_mode setting' (#1018) from yuka/garage:remove-replication-mode into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1018
2025-05-22 10:57:24 +00:00
Yureka
45bdf54e7e throw error on legacy replication_mode setting 2025-05-15 08:09:56 +02:00
Alex
a4b431163c Merge pull request 'speed up UploadPartCopy' (#1034) from yuka/garage:uploadpartcopy into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1034
2025-05-14 20:31:43 +00:00
Yureka
db54bf96c7 speed up UploadPartCopy 2025-05-14 02:12:41 +02:00
Yureka
cbcdab4e24 scripts: replication_mode -> replication_factor+consistency_mode 2025-05-04 16:43:56 +02:00
Alex
38ca35eb0f Merge pull request 'refactor: make TableShardedReplication a thin wrapper around LayoutManager' (#820) from yuka/garage:refactor-sharded-table into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/820
2025-04-28 10:43:35 +00:00
Yureka
a2d87a012d refactor: use replication factor of the layout versions in calculate_sync_map_min_with_quorum 2025-04-28 11:51:01 +02:00
Yureka
899292ee28 refactor: make TableShardedReplication a thin wrapper around LayoutManager 2025-04-28 11:51:01 +02:00
Yureka
c8e9c45889 refactor: Use ReplicationFactor type in more places
- Remove the replication_factor.replication_factor() in favor of
  usize::from(replication_factor) to make the conversion more explicit.

- Implement Display on ReplicationFactor so that it can be formatted
  without converting to usize

- Use ReplicationFactor in the constructor of LayoutVersion and add a
  method to get a ReplicationFactor from a LayoutVersion, despite
  LayoutVersion still storing it as usize internally.
2025-04-21 19:47:14 +02:00
Alex Auvolat
e79b485aa8 fix panic in ListAdminTokens 2025-04-17 17:38:20 +02:00
Alex Auvolat
d38d62f4d7 bump version to v2.0.0 2025-04-17 12:36:41 +02:00
Alex
2885806e00 Merge pull request 'creation and expiration dates' (#1010) from key-creation-expiration into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1010
2025-04-17 10:23:36 +00:00
Alex Auvolat
52437e4298 refactor parsing of --expires-in 2025-04-17 12:14:51 +02:00
Alex Auvolat
abcef7a3fd cli: implement garage key delete-expired 2025-04-17 11:58:19 +02:00
Alex Auvolat
5d338f0b8f add never_expires to remove expiration dates of admin tokens and access keys 2025-04-17 11:44:09 +02:00
Alex Auvolat
590c9bb4db possibility to update access key expiration date 2025-04-17 11:30:58 +02:00
Alex Auvolat
c56b7e20c3 add creation date and expiration date to access keys 2025-04-17 11:28:44 +02:00
Alex Auvolat
2f21181ccb publish bucket creation date in admin api and CLI 2025-04-17 10:29:23 +02:00
Alex
2d1c073d2f Merge pull request 'admin api: implement InspectObject (fix #892)' (#1005) from inspect-object into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1005
2025-04-06 12:33:11 +00:00
Alex Auvolat
5e7307cbf3 admin api: add comments for InspectObject 2025-04-06 14:21:54 +02:00
Alex Auvolat
fd0e23e984 admin api: implement InspectObject (fix #892) 2025-04-06 13:23:39 +02:00
Alex
d7506b282c Merge pull request 'admin api: always return latest bucket info' (#1004) from fix-bucket-info-api into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1004
2025-04-06 09:43:41 +00:00
Alex Auvolat
6bbdca2e48 admin api: always return latest bucket info 2025-04-06 11:14:42 +02:00
Alex
c6d6cc1fc3 Merge pull request 'read/write quorums on admin operations' (#997) from admin-quorums into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/997
2025-03-25 16:09:06 +00:00
Alex Auvolat
5fa6df6ee3 improve comments in bucket helper 2025-03-25 16:58:12 +01:00
Alex Auvolat
c6bed26347 relocalize logic into block manager 2025-03-25 16:41:19 +01:00
Alex Auvolat
d25e631a4a relocalize logic for write_sets 2025-03-25 16:35:56 +01:00
Alex Auvolat
514eb29874 use a WriteLock for write operations on fullcopy tables 2025-03-25 13:26:23 +01:00
Alex Auvolat
8ba6454e21 reduce anti-entropy interval for fullcopy tables 2025-03-25 13:11:11 +01:00
Alex Auvolat
9dcc5232a6 admin api: use fast local reads for token verification 2025-03-25 13:07:45 +01:00
Alex Auvolat
1e13a66b42 rework bucket helper functions to use local access where relevant 2025-03-25 13:00:48 +01:00
Alex Auvolat
2c9e849bbf remove dependency from garage_block to garage_table 2025-03-25 11:43:41 +01:00
Alex Auvolat
34baade499 fullcopy replication: quorum reads and writes 2025-03-25 11:40:09 +01:00
Alex Auvolat
2f2a96b51d layout & replication mode refactoring 2025-03-25 11:05:00 +01:00
Alex Auvolat
c9156f6828 Merge branch 'main' into next-v2 2025-03-25 10:02:41 +01:00
Alex
4629ee25f7 Merge pull request 'add crc64nvme checksumming algorithm (fix #963)' (#991) from add-crc64nvme into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/991
2025-03-20 12:28:35 +00:00
Alex Auvolat
a826c361a9 add crc64nvme checksumming algorithm (fix #963) 2025-03-19 15:51:06 +01:00
Alex
fb6db494cc Merge pull request 'sse-c: use different object encryption key for each object (fix #848)' (#990) from fix-ssec-crypto into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/990
2025-03-19 13:12:24 +00:00
Alex Auvolat
97e2fa5b8b add upgrade test for sse-c 2025-03-18 19:09:38 +01:00
Alex Auvolat
cfd259190f sse-c: use different object encryption key for each object 2025-03-18 16:23:37 +01:00
Alex
48e0436f29 Merge pull request 'cli: uniformize output and add some infos' (#984) from uniformize-cli into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/984
2025-03-12 15:51:11 +00:00
Alex Auvolat
9c745548c4 test-upgrade with v1 -> v2 2025-03-12 16:06:28 +01:00
Alex Auvolat
f7d9c2b383 cli: add garage json-api command and fix cargo tests 2025-03-12 15:55:52 +01:00
Alex Auvolat
e6862c5d3d cli: uniformize output and add some infos 2025-03-12 15:05:37 +01:00
Alex
d032e2017c Merge pull request 'admin api improvements' (#983) from admin-api-improvements into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/983
2025-03-12 11:00:24 +00:00
Alex Auvolat
0b12debf6c admin api: generate params from struct 2025-03-12 11:07:12 +01:00
Alex Auvolat
795b4a41b7 admin api: add special endpoints to openapi spec 2025-03-12 10:52:58 +01:00
Alex
fd2472d488 Merge pull request 'support for multiple admin API token' (#982) from multi-admin-token into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/982
2025-03-12 09:30:19 +00:00
Alex Auvolat
d2a064bb1b cli: add and remove scopes using --scope=+Scope or --scope=-Scope 2025-03-12 10:20:08 +01:00
Alex Auvolat
88b4623bf1 add creation date to admin api tokens 2025-03-12 09:57:32 +01:00
Alex Auvolat
325f79012c admin_token_table: implement is_tombstone() 2025-03-12 09:29:54 +01:00
Alex Auvolat
eb40475f1e move bucket search logic from helper to admin api 2025-03-12 09:21:53 +01:00
Alex Auvolat
22c0420607 admin api: specify date-time format in openapi spec 2025-03-11 18:25:46 +01:00
Alex Auvolat
1bd7689301 cli: add functions to manage admin api tokens 2025-03-11 18:09:24 +01:00
Alex Auvolat
ec0da3b644 admin api: mention admin_token and metrics_token in ListAdminTokensResponse 2025-03-11 15:57:29 +01:00
Alex Auvolat
9511b20153 admin api: add openapi spec for admin token management functions 2025-03-11 15:38:38 +01:00
Alex Auvolat
d067a40b3f admin api: add functions to manage admin api tokens 2025-03-11 15:17:31 +01:00
Alex Auvolat
ff6ec62d54 admin api: add metrics_require_token config option and update doc 2025-03-11 14:17:22 +01:00
Alex Auvolat
004eb94e14 admin api: verify tokens using the new admin api token table 2025-03-11 13:51:19 +01:00
Alex Auvolat
46f620119b add model for admin key table 2025-03-11 13:51:19 +01:00
Alex Auvolat
576d0d950e admin api: move functions to their correct location 2025-03-11 13:09:35 +01:00
Alex
85a07c87d7 Merge pull request 'admin refactoring, step 4' (#980) from refactor-admin into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/980
2025-03-11 09:19:12 +00:00
Alex Auvolat
1f645830a4 layout: make optional version mandatory in apply layout changes 2025-03-11 10:05:02 +01:00
Alex Auvolat
5f308bd688 move zone redundancy parsing/formatting to cli 2025-03-11 10:00:37 +01:00
Alex Auvolat
df758e8e0d cli v2: simplify 2025-03-11 09:54:05 +01:00
Alex Auvolat
e83864af24 layout: better encapsulation 2025-03-11 09:29:38 +01:00
Alex Auvolat
3b49dd9e63 admin api: small refactor + add comments to layout-related calls 2025-03-11 09:19:20 +01:00
Alex Auvolat
cef8d75983 admin api: avoid overwriting redirect rules in UpdateBucket 2025-03-07 16:25:55 +01:00
Alex Auvolat
cd0728cd20 cli: move files around 2025-03-06 18:54:40 +01:00
Alex Auvolat
0951b5db75 admin api: implement ClusterLayoutSkipDeadNodes and use it in CLI 2025-03-06 18:49:56 +01:00
Alex Auvolat
3d94eb8d4b admin api: implement GetClusterLayoutHistory and use it in CLI 2025-03-06 18:33:05 +01:00
Alex Auvolat
004866caac admin api, cliv2: implement garage layout show using api functions 2025-03-06 17:56:22 +01:00
Alex Auvolat
913e6da41b admin api: implement PreviewClusterLayoutChanges 2025-03-06 17:27:23 +01:00
Alex Auvolat
e4881e62f1 admin api: management of layout parameters through admin api 2025-03-06 17:12:52 +01:00
Alex
7ccbfda26d Merge pull request 'admin api: generate openapi spec using utoipa' (#979) from refactor-admin into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/979
2025-03-06 15:09:00 +00:00
Alex Auvolat
6b19d7628e admin api: small fixes and reordering 2025-03-06 14:21:25 +01:00
Alex Auvolat
411f1d495c admin api: add all missing endpoints to openapi spec 2025-03-06 14:01:39 +01:00
Alex Auvolat
ba68506c36 admin api: generate openapi spec using utoipa (wip) 2025-03-06 12:26:14 +01:00
Alex
21c83ab311 Merge pull request 'admi api: remove info about local node from GetClusterStatus and add specific GetNodeInfo endpoint' (#978) from refactor-admin into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/978
2025-03-06 09:50:22 +00:00
Alex Auvolat
2e03d90585 admi api: remove info about local node from GetClusterStatus and add specific GetNodeInfo endpoint 2025-03-06 10:26:01 +01:00
Alex Auvolat
29ce490dd6 Merge branch 'main' into next-v2 2025-03-05 14:50:22 +01:00
Alex
c3e8e5e38c Merge pull request 'support redirection on s3 endpoint' (#910) from trinity-1686a/garage:1686a/s3-redirects into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/910
2025-02-14 12:58:48 +00:00
Alex Auvolat
62a3003cca rename Condition into RedirectCondition in internal model 2025-02-14 13:45:32 +01:00
Alex Auvolat
3151695011 Merge branch 'next-v2' into 1686a/s3-redirects 2025-02-14 13:11:33 +01:00
trinity-1686a
f034e834fa Merge branch 'main' into 1686a/s3-redirects 2025-02-08 16:01:11 +01:00
trinity-1686a
bf0f792418 add redirect_all to WebsiteConfig model 2025-02-08 15:36:48 +01:00
Alex Auvolat
61f3de6496 Merge branch 'main' into next-v2 2025-02-05 21:00:19 +01:00
Alex
71655c1e89 Merge pull request 'Admin API refactoring: convert existing commands to API requests (step 3)' (#945) from refactor-admin into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/945
2025-02-05 19:54:42 +00:00
Alex Auvolat
7c8fc04b96 massively speed up compilation of garage_api_admin by not using async_trait 2025-02-05 19:37:38 +01:00
Alex Auvolat
f914db057a cli_v2: implement LaunchRepairOperation and remove old stuff 2025-02-05 15:36:47 +01:00
Alex Auvolat
406b6da163 cli_v2: implement Get{Node,Cluster}Statistics 2025-02-05 15:06:10 +01:00
Alex Auvolat
9f468b4439 cli_v2: implement CreateMetadataSnapshot 2025-02-05 14:22:10 +01:00
Alex Auvolat
97be7b38fa admin api: reorder things 2025-02-03 18:54:51 +01:00
Alex Auvolat
6a1079c412 admin api: impl RequestHandler for MetricsRequest 2025-02-03 18:54:51 +01:00
Alex Auvolat
b1629dd355 cli_v2: implement RetryBlockResync and PurgeBlocks 2025-02-03 18:54:51 +01:00
Alex Auvolat
d405a9f839 cli_v2: implement ListBlockErrors and GetBlockInfo 2025-02-03 18:54:51 +01:00
Alex Auvolat
7b9c047b11 cli_v2: add local_api_request with crazy type bound 2025-02-03 18:54:51 +01:00
Alex Auvolat
10bbb26b30 cli_v2: implement ListWorkers and GetWorkerInfo 2025-02-03 18:54:51 +01:00
Alex Auvolat
89ff9f5576 admin api: base infrastructure for local endpoints
admin api: rename EndpointHandler into RequestHandler to avoid confusion with RPC

wip: infrastructure for local api calls

admin api: fix things

admin api: first local endpoint to work with new scheme

admin api: implement SetWorkerVariable
2025-02-03 18:54:51 +01:00
Alex Auvolat
bdaf55ab3f cli_v2: migrate cleanupincompleteuploads to Admin API
admin api: add CleanupIncompleteUploads spec
2025-02-03 18:54:51 +01:00
Alex Auvolat
e96014ca60 Merge branch 'main' into next-v2 2025-02-03 18:53:17 +01:00
Alex Auvolat
568c4954e9 Merge branch 'main' into next-v2 2025-02-01 19:58:36 +01:00
Alex Auvolat
fe937c2901 Merge branch 'main' into next-v2 2025-02-01 19:07:17 +01:00
Alex
3192088aac Merge pull request 'admin refactoring: refactor CLI to use Admin API requests (step 2)' (#943) from refactor-admin into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/943
2025-01-30 16:18:38 +00:00
Alex Auvolat
5a89350b38 cli_v2: fix garage status 2025-01-30 16:45:59 +01:00
Alex Auvolat
3caea5fc06 cli_v2: merge util.rs into layout.rs 2025-01-30 16:24:55 +01:00
Alex Auvolat
ebc0e9319e cli_v2: error messages 2025-01-30 16:17:35 +01:00
Alex Auvolat
f8c6a8373d convert cli key operations to admin rpc 2025-01-30 16:12:16 +01:00
Alex Auvolat
076ce04fe5 fix garage status output 2025-01-30 15:38:22 +01:00
Alex Auvolat
f37d5d2b08 admin api: convert most bucket operations 2025-01-30 13:36:25 +01:00
Alex Auvolat
819f4f0050 cli: migrate layout remove, apply, revert 2025-01-30 12:19:23 +01:00
Alex Auvolat
69ddaafc60 wip: migrate garage status and garage layout assign 2025-01-30 12:07:12 +01:00
Alex Auvolat
145130481e wip: proxy admin api requests through admin rpc, prepare new cli 2025-01-30 10:44:08 +01:00
Alex
6ed78abb5c Merge pull request 'admin API refactoring (step 1)' (#939) from refactor-admin into next-v2
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/939
2025-01-29 20:42:56 +00:00
Alex Auvolat
19454c1679 admin api: remove log message 2025-01-29 19:47:37 +01:00
Alex Auvolat
1c03941b19 admin api: fix panic on GetKeyInfo with no args 2025-01-29 19:26:16 +01:00
Alex Auvolat
4f0b923c4f admin api: small fixes 2025-01-29 19:26:16 +01:00
Alex Auvolat
420bbc162d admin api: clearer syntax for AddBucketAlias and RemoveBucketAlias 2025-01-29 19:26:16 +01:00
Alex Auvolat
12ea4cda5f admin api: merge calls to manage global/local aliases 2025-01-29 19:26:16 +01:00
Alex Auvolat
5fefbd94e9 admin api: rename allow/deny api calls in api v2 2025-01-29 19:26:16 +01:00
Alex Auvolat
ba810b2e81 admin api: rename bucket aliasing operations 2025-01-29 19:26:16 +01:00
Alex Auvolat
f8ed3fdbc4 fix test_website_check_domain 2025-01-29 19:26:16 +01:00
Alex Auvolat
2daeb89834 admin api: fixes to openapi v2 spec 2025-01-29 19:26:16 +01:00
Alex Auvolat
4cb45bd398 admin api: fix CORS to work in browser 2025-01-29 19:26:16 +01:00
Alex Auvolat
d5ad797ad7 admin api: update v2 openapi spec 2025-01-29 19:26:16 +01:00
Alex Auvolat
a99925e0ed admin api: initialize v2 openapi spec from v1 2025-01-29 19:26:16 +01:00
Alex Auvolat
f538dc34d3 admin api: make all requests and responses (de)serializable 2025-01-29 19:26:16 +01:00
Alex Auvolat
ed58f8b0fe admin api: update semantics of some endpoints, and update doc 2025-01-29 19:26:16 +01:00
Alex Auvolat
5037b97dd4 admin api: add compatibility from v1/ to v2/ 2025-01-29 19:26:16 +01:00
Alex Auvolat
af1a530834 admin api: refactor using macro 2025-01-29 19:26:16 +01:00
Alex Auvolat
c99bfe69ea admin api: new router_v2 with unified path syntax 2025-01-29 19:26:16 +01:00
Alex Auvolat
831f2b0207 admin api: make all handlers impls of a single trait 2025-01-29 19:26:16 +01:00
Alex Auvolat
c1eb1610ba admin api: create structs for all requests/responess in src/api/admin/api.rs 2025-01-29 19:26:16 +01:00
Quentin Dufour
5560a963e0 decrease write quorum 2025-01-29 19:25:58 +01:00
Alex Auvolat
2aaba39ddc refactor web_server.rs 2025-01-04 20:11:54 +01:00
Alex Auvolat
47467df83e avoid handling status_code-related logic in api/s3/get.rs 2025-01-04 19:52:14 +01:00
Alex Auvolat
9b7fea4cb0 put bucket website: improve error message for redirectallrequests 2025-01-04 19:16:24 +01:00
Alex Auvolat
44ce6ae5b4 properly implement new bucket model using a migration 2025-01-04 18:50:49 +01:00
Alex Auvolat
22487ceddf move Redirect::compute_target to standalone function in web_server.rs 2025-01-04 18:22:42 +01:00
Alex Auvolat
6ccfbb2986 remove obsolete RedirectAll struct 2025-01-04 17:04:17 +01:00
trinity-1686a
c939d2a936 clippy 2024-12-22 15:26:06 +01:00
trinity-1686a
65e9dde8c9 add tests 2024-12-22 15:20:09 +01:00
trinity-1686a
c9b733a4a6 support redirection on s3 endpoint 2024-12-14 17:46:27 +01:00
256 changed files with 18362 additions and 6183 deletions

View file

@ -16,6 +16,16 @@ steps:
commands:
- nix-build -j4 --attr flakePackages.fmt
- name: check typos
image: nixpkgs/nix:nixos-24.05
commands:
- nix-shell --attr ci --run typos
- name: check lints with clippy
image: nixpkgs/nix:nixos-24.05
commands:
- nix-build -j4 --attr flakePackages.clippy
- name: build
image: nixpkgs/nix:nixos-24.05
commands:

View file

@ -38,7 +38,15 @@ steps:
- matrix:
ARCH: i386
- name: upgrade tests
- name: upgrade tests from v1.0.0
image: nixpkgs/nix:nixos-24.05
commands:
- nix-shell --attr ci --run "./script/test-upgrade.sh v1.0.0 x86_64-unknown-linux-musl" || (cat /tmp/garage.log; false)
when:
- matrix:
ARCH: amd64
- name: upgrade tests from v0.8.4
image: nixpkgs/nix:nixos-24.05
commands:
- nix-shell --attr ci --run "./script/test-upgrade.sh v0.8.4 x86_64-unknown-linux-musl" || (cat /tmp/garage.log; false)

152
Cargo.lock generated
View file

@ -301,9 +301,9 @@ dependencies = [
[[package]]
name = "aws-sdk-s3"
version = "1.120.0"
version = "1.102.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "06673901e961f20fa8d7da907da48f7ad6c1b383e3726c22bd418900f015abe1"
checksum = "75ddb925e840f49446aa6338b67abdbec04b4ebf923b7da038ec4c35afb916cd"
dependencies = [
"aws-credential-types",
"aws-runtime",
@ -313,7 +313,6 @@ dependencies = [
"aws-smithy-eventstream",
"aws-smithy-http",
"aws-smithy-json",
"aws-smithy-observability",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
"aws-smithy-types",
@ -370,9 +369,9 @@ dependencies = [
[[package]]
name = "aws-smithy-checksums"
version = "0.63.13"
version = "0.63.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "23374b9170cbbcc6f5df8dc5ebb9b6c5c28a3c8f599f0e8b8b10eb6f4a5c6e74"
checksum = "9054b4cc5eda331cde3096b1576dec45365c5cbbca61d1fffa5f236e251dfce7"
dependencies = [
"aws-smithy-http",
"aws-smithy-types",
@ -865,42 +864,16 @@ dependencies = [
"libc",
]
[[package]]
name = "crc"
version = "3.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9710d3b3739c2e349eb44fe848ad0b7c8cb1e42bd87ee49371df2f7acaf3e675"
dependencies = [
"crc-catalog",
]
[[package]]
name = "crc-catalog"
version = "2.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19d374276b40fb8bbdee95aef7c7fa6b5316ec764510eb64b8dd0e2ed0d7e7f5"
[[package]]
name = "crc-fast"
version = "1.9.0"
version = "1.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2fd92aca2c6001b1bf5ba0ff84ee74ec8501b52bbef0cac80bf25a6c1d87a83d"
checksum = "e75b2483e97a5a7da73ac68a05b629f9c53cff58d8ed1c77866079e18b00dba5"
dependencies = [
"crc",
"digest",
"rustversion",
"spin 0.10.0",
]
[[package]]
name = "crc32c"
version = "0.6.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a47af21622d091a8f0fb295b88bc886ac74efcc613efc19f5d0b21de5c89e47"
dependencies = [
"rustc_version",
]
[[package]]
name = "crc32fast"
version = "1.5.0"
@ -1201,12 +1174,6 @@ version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"
[[package]]
name = "foldhash"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb"
[[package]]
name = "form_urlencoded"
version = "1.2.2"
@ -1311,7 +1278,7 @@ dependencies = [
[[package]]
name = "garage"
version = "1.3.1"
version = "2.2.0"
dependencies = [
"assert-json-diff",
"async-trait",
@ -1322,7 +1289,7 @@ dependencies = [
"bytes",
"bytesize",
"chrono",
"crc32fast",
"crc-fast",
"format_table",
"futures",
"garage_api_admin",
@ -1351,7 +1318,6 @@ dependencies = [
"opentelemetry-otlp",
"opentelemetry-prometheus",
"parse_duration",
"serde",
"serde_json",
"sha1",
"sha2",
@ -1363,16 +1329,21 @@ dependencies = [
"tracing",
"tracing-journald",
"tracing-subscriber",
"utoipa",
]
[[package]]
name = "garage_api_admin"
version = "1.3.1"
version = "2.2.0"
dependencies = [
"argon2",
"async-trait",
"bytesize",
"chrono",
"format_table",
"futures",
"garage_api_common",
"garage_block",
"garage_model",
"garage_rpc",
"garage_table",
@ -1382,6 +1353,7 @@ dependencies = [
"hyper 1.8.1",
"opentelemetry",
"opentelemetry-prometheus",
"paste",
"prometheus",
"serde",
"serde_json",
@ -1389,17 +1361,17 @@ dependencies = [
"tokio",
"tracing",
"url",
"utoipa",
]
[[package]]
name = "garage_api_common"
version = "1.3.1"
version = "2.2.0"
dependencies = [
"base64 0.21.7",
"bytes",
"chrono",
"crc32c",
"crc32fast",
"crc-fast",
"crypto-common",
"futures",
"garage_model",
@ -1427,7 +1399,7 @@ dependencies = [
[[package]]
name = "garage_api_k2v"
version = "1.3.1"
version = "2.2.0"
dependencies = [
"base64 0.21.7",
"futures",
@ -1450,15 +1422,14 @@ dependencies = [
[[package]]
name = "garage_api_s3"
version = "1.3.1"
version = "2.2.0"
dependencies = [
"aes-gcm",
"async-compression",
"base64 0.21.7",
"bytes",
"chrono",
"crc32c",
"crc32fast",
"crc-fast",
"form_urlencoded",
"futures",
"garage_api_common",
@ -1469,6 +1440,7 @@ dependencies = [
"garage_table",
"garage_util",
"hex",
"hmac",
"http 1.4.0",
"http-body-util",
"http-range",
@ -1495,7 +1467,7 @@ dependencies = [
[[package]]
name = "garage_block"
version = "1.3.1"
version = "2.2.0"
dependencies = [
"arc-swap",
"async-compression",
@ -1506,7 +1478,6 @@ dependencies = [
"garage_db",
"garage_net",
"garage_rpc",
"garage_table",
"garage_util",
"hex",
"opentelemetry",
@ -1520,7 +1491,7 @@ dependencies = [
[[package]]
name = "garage_db"
version = "1.3.1"
version = "2.2.0"
dependencies = [
"fjall",
"heed",
@ -1535,8 +1506,9 @@ dependencies = [
[[package]]
name = "garage_model"
version = "1.3.1"
version = "2.2.0"
dependencies = [
"argon2",
"async-trait",
"base64 0.21.7",
"blake2",
@ -1562,7 +1534,7 @@ dependencies = [
[[package]]
name = "garage_net"
version = "1.3.1"
version = "2.2.0"
dependencies = [
"arc-swap",
"bytes",
@ -1587,7 +1559,7 @@ dependencies = [
[[package]]
name = "garage_rpc"
version = "1.3.1"
version = "2.2.0"
dependencies = [
"arc-swap",
"async-trait",
@ -1619,7 +1591,7 @@ dependencies = [
[[package]]
name = "garage_table"
version = "1.3.1"
version = "2.2.0"
dependencies = [
"arc-swap",
"async-trait",
@ -1640,7 +1612,7 @@ dependencies = [
[[package]]
name = "garage_util"
version = "1.3.1"
version = "2.2.0"
dependencies = [
"arc-swap",
"async-trait",
@ -1672,14 +1644,13 @@ dependencies = [
[[package]]
name = "garage_web"
version = "1.3.1"
version = "2.2.0"
dependencies = [
"garage_api_common",
"garage_api_s3",
"garage_model",
"garage_table",
"garage_util",
"html-escape",
"http 1.4.0",
"http-body-util",
"hyper 1.8.1",
@ -1835,7 +1806,9 @@ version = "0.15.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
dependencies = [
"foldhash 0.1.5",
"allocator-api2",
"equivalent",
"foldhash",
]
[[package]]
@ -1843,11 +1816,6 @@ name = "hashbrown"
version = "0.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100"
dependencies = [
"allocator-api2",
"equivalent",
"foldhash 0.2.0",
]
[[package]]
name = "hashlink"
@ -1950,15 +1918,6 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "html-escape"
version = "0.2.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d1ad449764d627e22bfd7cd5e8868264fc9236e07c752972b4080cd351cb476"
dependencies = [
"utf8-width",
]
[[package]]
name = "http"
version = "0.2.12"
@ -2314,6 +2273,8 @@ checksum = "7714e70437a7dc3ac8eb7e6f8df75fd8eb422675fc7678aff7364301092b1017"
dependencies = [
"equivalent",
"hashbrown 0.16.1",
"serde",
"serde_core",
]
[[package]]
@ -2692,11 +2653,11 @@ checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897"
[[package]]
name = "lru"
version = "0.16.3"
version = "0.12.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a1dc47f592c06f33f8e3aea9591776ec7c9f9e4124778ff8a3c3b87159f7e593"
checksum = "234cf4f4a04dc1f57e24b96cc0cd600cf2af460d4161ac5ecdd0af8e1f3b2a38"
dependencies = [
"hashbrown 0.16.1",
"hashbrown 0.15.5",
]
[[package]]
@ -3118,6 +3079,12 @@ dependencies = [
"subtle",
]
[[package]]
name = "paste"
version = "1.0.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a"
[[package]]
name = "path-absolutize"
version = "3.1.1"
@ -4830,12 +4797,6 @@ dependencies = [
"serde",
]
[[package]]
name = "utf8-width"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1292c0d970b54115d14f2492fe0170adf21d68a1de108eebc51c1df4f346a091"
[[package]]
name = "utf8_iter"
version = "1.0.4"
@ -4848,6 +4809,29 @@ version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "utoipa"
version = "5.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2fcc29c80c21c31608227e0912b2d7fddba57ad76b606890627ba8ee7964e993"
dependencies = [
"indexmap 2.13.0",
"serde",
"serde_json",
"utoipa-gen",
]
[[package]]
name = "utoipa-gen"
version = "5.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d79d08d92ab8af4c5e8a6da20c47ae3f61a0f1dabc1997cdf2d082b757ca08b"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.114",
]
[[package]]
name = "uuid"
version = "1.4.1"

View file

@ -24,22 +24,22 @@ default-members = ["src/garage"]
# Internal Garage crates
format_table = { version = "0.1.1", path = "src/format-table" }
garage_api_common = { version = "1.3.1", path = "src/api/common" }
garage_api_admin = { version = "1.3.1", path = "src/api/admin" }
garage_api_s3 = { version = "1.3.1", path = "src/api/s3" }
garage_api_k2v = { version = "1.3.1", path = "src/api/k2v" }
garage_block = { version = "1.3.1", path = "src/block" }
garage_db = { version = "1.3.1", path = "src/db", default-features = false }
garage_model = { version = "1.3.1", path = "src/model", default-features = false }
garage_net = { version = "1.3.1", path = "src/net" }
garage_rpc = { version = "1.3.1", path = "src/rpc" }
garage_table = { version = "1.3.1", path = "src/table" }
garage_util = { version = "1.3.1", path = "src/util" }
garage_web = { version = "1.3.1", path = "src/web" }
garage_api_common = { version = "2.2.0", path = "src/api/common" }
garage_api_admin = { version = "2.2.0", path = "src/api/admin" }
garage_api_s3 = { version = "2.2.0", path = "src/api/s3" }
garage_api_k2v = { version = "2.2.0", path = "src/api/k2v" }
garage_block = { version = "2.2.0", path = "src/block" }
garage_db = { version = "2.2.0", path = "src/db", default-features = false }
garage_model = { version = "2.2.0", path = "src/model", default-features = false }
garage_net = { version = "2.2.0", path = "src/net" }
garage_rpc = { version = "2.2.0", path = "src/rpc" }
garage_table = { version = "2.2.0", path = "src/table" }
garage_util = { version = "2.2.0", path = "src/util" }
garage_web = { version = "2.2.0", path = "src/web" }
k2v-client = { version = "0.0.4", path = "src/k2v-client" }
# External crates from crates.io
arc-swap = "1.0"
arc-swap = "1.1"
argon2 = "0.5"
async-trait = "0.1.7"
backtrace = "0.3"
@ -48,15 +48,13 @@ blake2 = "0.10"
bytes = "1.0"
bytesize = "1.1"
cfg-if = "1.0"
chrono = "0.4"
crc32fast = "1.4"
crc32c = "0.6"
chrono = { version = "0.4", features = ["serde"] }
crc-fast = "1.6"
crypto-common = "0.1"
gethostname = "0.4"
git-version = "0.3.4"
hex = "0.4"
hexdump = "0.1"
html-escape = "0.2.13"
hmac = "0.12"
itertools = "0.12"
ipnet = "2.9.0"
@ -67,6 +65,7 @@ nix = { version = "0.29", default-features = false, features = ["fs"] }
nom = "7.1"
parking_lot = "0.12"
parse_duration = "2.1"
paste = "1.0"
pin-project = "1.0.12"
pnet_datalink = "0.34"
rand = "0.8"
@ -96,12 +95,13 @@ fjall = "2.4"
async-compression = { version = "0.4", features = ["tokio", "zstd"] }
zstd = { version = "0.13", default-features = false }
quick-xml = { version = "0.26", features = [ "serialize" ] }
quick-xml = { version = "0.26", features = ["serialize"] }
rmp-serde = "1.1.2"
serde = { version = "1.0", default-features = false, features = ["derive", "rc"] }
serde_bytes = "0.11"
serde_json = "1.0"
toml = { version = "0.8", default-features = false, features = ["parse"] }
utoipa = { version = "5.3.1", features = ["chrono"] }
# newer version requires rust edition 2021
k8s-openapi = { version = "0.21", features = ["v1_24"] }
@ -115,7 +115,7 @@ httpdate = "1.0"
http-range = "0.1"
http-body-util = "0.1"
hyper = { version = "1.0", default-features = false }
hyper-util = { version = "0.1", features = [ "full" ] }
hyper-util = { version = "0.1", features = ["full"] }
multer = "3.0"
percent-encoding = "2.2"
roxmltree = "0.19"
@ -123,11 +123,11 @@ url = "2.3"
futures = "0.3"
futures-util = "0.3"
tokio = { version = "1.0", default-features = false, features = ["net", "rt", "rt-multi-thread", "io-util", "net", "time", "macros", "sync", "signal", "fs"] }
tokio = { version = "1.0", default-features = false, features = ["rt", "rt-multi-thread", "io-util", "net", "time", "macros", "sync", "signal", "fs"] }
tokio-util = { version = "0.7", features = ["compat", "io"] }
tokio-stream = { version = "0.1", features = ["net"] }
opentelemetry = { version = "0.17", features = [ "rt-tokio", "metrics", "trace" ] }
opentelemetry = { version = "0.17", features = ["rt-tokio", "metrics", "trace"] }
opentelemetry-prometheus = "0.10"
opentelemetry-otlp = "0.10"
opentelemetry-contrib = "0.9"
@ -147,8 +147,12 @@ aws-smithy-runtime = { version = "1.8", default-features = false, features = ["t
aws-sdk-config = { version = "1.62", default-features = false }
aws-sdk-s3 = { version = "1.79", default-features = false, features = ["rt-tokio"] }
[profile.dev]
#lto = "thin" # disabled for now, adds 2-4 min to each CI build
lto = "off"
[profile.release]
lto = "thin"
codegen-units = 16
lto = true
codegen-units = 1
opt-level = 3
strip = "debuginfo"
strip = true

View file

@ -1,7 +1,7 @@
<!DOCTYPE html>
<html>
<head>
<title>Garage Adminstration API v0</title>
<title>Garage administration API v0</title>
<!-- needed for adaptive design -->
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1">

View file

@ -3,10 +3,10 @@ info:
version: v0.8.0
title: Garage Administration API v0+garage-v0.8.0
description: |
Administrate your Garage cluster programatically, including status, layout, keys, buckets, and maintainance tasks.
*Disclaimer: The API is not stable yet, hence its v0 tag. The API can change at any time, and changes can include breaking backward compatibility. Read the changelog and upgrade your scripts before upgrading. Additionnaly, this specification is very early stage and can contain bugs, especially on error return codes/types that are not tested yet. Do not expect a well finished and polished product!*
paths:
Administrate your Garage cluster programmatically, including status, layout, keys, buckets, and maintenance tasks.
*Disclaimer: The API is not stable yet, hence its v0 tag. The API can change at any time, and changes can include breaking backward compatibility. Read the changelog and upgrade your scripts before upgrading. Additionally, this specification is very early stage and can contain bugs, especially on error return codes/types that are not tested yet. Do not expect a well finished and polished product!*
paths:
/status:
get:
tags:

View file

@ -1,7 +1,7 @@
<!DOCTYPE html>
<html>
<head>
<title>Garage Adminstration API v0</title>
<title>Garage administration API v1</title>
<!-- needed for adaptive design -->
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1">

View file

@ -3,10 +3,10 @@ info:
version: v0.9.0
title: Garage Administration API v0+garage-v0.9.0
description: |
Administrate your Garage cluster programatically, including status, layout, keys, buckets, and maintainance tasks.
*Disclaimer: The API is not stable yet, hence its v0 tag. The API can change at any time, and changes can include breaking backward compatibility. Read the changelog and upgrade your scripts before upgrading. Additionnaly, this specification is very early stage and can contain bugs, especially on error return codes/types that are not tested yet. Do not expect a well finished and polished product!*
paths:
Administrate your Garage cluster programmatically, including status, layout, keys, buckets, and maintenance tasks.
*Disclaimer: The API is not stable yet, hence its v0 tag. The API can change at any time, and changes can include breaking backward compatibility. Read the changelog and upgrade your scripts before upgrading. Additionally, this specification is very early stage and can contain bugs, especially on error return codes/types that are not tested yet. Do not expect a well finished and polished product!*
paths:
/health:
get:
tags:
@ -440,7 +440,7 @@ paths:
- "false"
example: "true"
required: false
description: "Wether or not the secret key should be returned in the response"
description: "Whether or not the secret key should be returned in the response"
responses:
'500':
description: "The server can not handle your request. Check your connectivity with the rest of the cluster."

View file

@ -0,0 +1,24 @@
<!DOCTYPE html>
<html>
<head>
<title>Garage administration API v2</title>
<!-- needed for adaptive design -->
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link href="./css/redoc.css" rel="stylesheet">
<!--
Redoc doesn't change outer page styles
-->
<style>
body {
margin: 0;
padding: 0;
}
</style>
</head>
<body>
<redoc spec-url='./garage-admin-v2.json'></redoc>
<script src="./redoc.standalone.js"> </script>
</body>
</html>

4430
doc/api/garage-admin-v2.json Normal file

File diff suppressed because it is too large Load diff

View file

@ -51,4 +51,4 @@ We are currently building this SDK for [Python](@/documentation/build/python.md#
More information:
- [In the reference manual](@/documentation/reference-manual/admin-api.md)
- [Full specifiction](https://garagehq.deuxfleurs.fr/api/garage-admin-v0.html)
- [Full specification](https://garagehq.deuxfleurs.fr/api/garage-admin-v0.html)

View file

@ -5,13 +5,13 @@ weight = 99
## S3
If you are developping a new application, you may want to use Garage to store your user's media.
If you are developing a new application, you may want to use Garage to store your user's media.
The S3 API that Garage uses is a standard REST API, so as long as you can make HTTP requests,
you can query it. You can check the [S3 REST API Reference](https://docs.aws.amazon.com/AmazonS3/latest/API/API_Operations_Amazon_Simple_Storage_Service.html) from Amazon to learn more.
Developping your own wrapper around the REST API is time consuming and complicated.
Instead, there are some libraries already avalaible.
Developing your own wrapper around the REST API is time consuming and complicated.
Instead, there are some libraries already available.
Some of them are maintained by Amazon, some by Minio, others by the community.

View file

@ -23,7 +23,7 @@ To configure S3-compatible software to interact with Garage,
you will need the following parameters:
- An **API endpoint**: this corresponds to the HTTP or HTTPS address
used to contact the Garage server. When runing Garage locally this will usually
used to contact the Garage server. When running Garage locally this will usually
be `http://127.0.0.1:3900`. In a real-world setting, you would usually have a reverse-proxy
that adds TLS support and makes your Garage server available under a public hostname
such as `https://garage.example.com`.

View file

@ -12,8 +12,9 @@ In this section, we cover the following web applications:
| [Mastodon](#mastodon) | ✅ | Natively supported |
| [Matrix](#matrix) | ✅ | Tested with `synapse-s3-storage-provider` |
| [ejabberd](#ejabberd) | ✅ | `mod_s3_upload` |
| [Pixelfed](#pixelfed) | ✅ | Natively supported |
| [Pleroma](#pleroma) | ❓ | Not yet tested |
| [Ente](#ente) | ✅ | Natively supported |
| [Pixelfed](#pixelfed) | ❓ | Natively supported |
| [Pleroma](#pleroma) | ✅ | Natively supported |
| [Lemmy](#lemmy) | ✅ | Supported with pict-rs |
| [Funkwhale](#funkwhale) | ❓ | Not yet tested |
| [Misskey](#misskey) | ❓ | Not yet tested |
@ -53,7 +54,7 @@ garage bucket allow nextcloud --read --write --key nextcloud-key
Now edit your Nextcloud configuration file to enable object storage.
On my installation, the config. file is located at the following path: `/var/www/nextcloud/config/config.php`.
We will add a new root key to the `$CONFIG` dictionnary named `objectstore`:
We will add a new root key to the `$CONFIG` dictionary named `objectstore`:
```php
<?php
@ -412,7 +413,7 @@ mc mirror --newer-than "3h" ./public/system/ garage/mastodon-data
## Matrix
Matrix is a chat communication protocol. Its main stable server implementation, [Synapse](https://matrix-org.github.io/synapse/latest/), provides a module to store media on a S3 backend. Additionally, a server independent media store supporting S3 has been developped by the community, it has been made possible thanks to how the matrix API has been designed and will work with implementations like Conduit, Dendrite, etc.
Matrix is a chat communication protocol. Its main stable server implementation, [Synapse](https://matrix-org.github.io/synapse/latest/), provides a module to store media on a S3 backend. Additionally, a server independent media store supporting S3 has been developed by the community, it has been made possible thanks to how the matrix API has been designed and will work with implementations like Conduit, Dendrite, etc.
### synapse-s3-storage-provider (synapse only)
@ -449,7 +450,7 @@ media_storage_providers:
Note that uploaded media will also be stored locally and this behavior can not be deactivated, it is even required for
some operations like resizing images.
In fact, your local filesysem is considered as a cache but without any automated way to garbage collect it.
In fact, your local filesystem is considered as a cache but without any automated way to garbage collect it.
We can build our garbage collector with `s3_media_upload`, a tool provided with the module.
If you installed the module with the command provided before, you should be able to bring it in your path:
@ -567,13 +568,186 @@ The module can then be configured with:
Other configuration options can be found in the
[configuration YAML file](https://github.com/processone/ejabberd-contrib/blob/master/mod_s3_upload/conf/mod_s3_upload.yml).
## Ente
Ente is an alternative for Google Photos and Apple Photos. It [can be selfhosted](https://help.ente.io/self-hosting/) and is working fine with Garage as of May 2024.
As a first step we need to create a bucket and a key for Ente:
```bash
garage bucket create ente
garage key create ente-key
# For the CORS setup to work, the key needs to be --owner as well, at least temporarily.
garage bucket allow ente --read --write --owner --key ente-key
```
We also need to setup some CORS rules to allow the Ente frontend to access the bucket:
```bash
export CORS='{"CORSRules":[{"AllowedHeaders":["*"],"AllowedMethods":["GET", "PUT", "POST", "DELETE"],"AllowedOrigins":["*"], "ExposeHeaders":["ETag"]}]}'
aws s3api put-bucket-cors --bucket ente --cors-configuration $CORS
```
Now we need to configure ente-server to use our bucket. This is explained [in the Ente S3 documentation](https://help.ente.io/self-hosting/guides/external-s3).
Prepare a configuration file for ente's backend as `museum.yaml`:
```yaml
credentials-file: /credentials.yaml
apps:
public-albums: https://albums.example.tld # If you want to use the share album feature
internal:
hardcoded-ott:
local-domain-suffix: "@example.com" # Your domain
local-domain-value: 123456 # Custom One-Time Password since we are not sending mail by default
key:
# WARNING -- You MUST CHANGE the values below
# Someone has made an image that can do it for you : https://github.com/EdyTheCow/ente-selfhost/blob/main/images/ente-server-tools/Dockerfile
# Simply build it yourself or run docker run --rm ghcr.io/edythecow/ente-server-tools go run tools/gen-random-keys/main.go
encryption: yvmG/RnzKrbCb9L3mgsmoxXr9H7i2Z4qlbT0mL3ln4w= # CHANGE THIS VALUE
hash: KXYiG07wC7GIgvCSdg+WmyWdXDAn6XKYJtp/wkEU7x573+byBRAYtpTP0wwvi8i/4l37uicX1dVTUzwH3sLZyw== # CHANGE THIS VALUE
jwt:
secret: i2DecQmfGreG6q1vBj5tCokhlN41gcfS2cjOs9Po-u8= # CHANGE THIS VALUE
```
The full configuration file can be found [here](https://github.com/ente-io/ente/blob/main/server/configurations/local.yaml)
Then prepare a credentials file as `credentials.yaml`
```yaml
db:
host: postgres
port: 5432
name: <ente_db_name>
user: <pguser>
password: <pgpass>
s3:
# Override the primary and secondary hot storage. The commented out values
# are the defaults.
#
hot_storage:
primary: b2-eu-cen
# secondary: wasabi-eu-central-2-v3
# If true, enable some workarounds to allow us to use a local minio instance
# for object storage.
#
# 1. Disable SSL.
# 2. Use "path" style S3 URLs (see `use_path_style_urls` below).
# 3. Directly download the file during replication instead of going via the
# Cloudflare worker.
# 4. Do not specify storage classes when uploading objects (since minio does
# not support them, specifically it doesn't support GLACIER).
are_local_buckets: true
# To use "path" style S3 URLs instead of DNS-based bucket access
# default to true if you set "are_local_buckets: true"
# use_path_style_urls: true
b2-eu-cen: # Don't change this key, it is hardcoded
key: <keyID>
secret: <keySecret>
endpoint: garage:3900 # publicly accessible endpoint of your garage instance
region: garage
bucket: <yourbucketName>
use_path_style: true
# you can specify secondary locations, names are hardcoded as well
# wasabi-eu-central-2-v3:
# scw-eu-fr-v3:
# and you can also specify a bucket to be used for embeddings, preview etc..
# default to the first bucket
# derived-storage: wasabi-eu-central-2-derived
```
Finally you can run it with Docker :
```bash
docker run -d --name ente-server --restart unless-stopped -v /path/to/museum.yaml:/museum.yaml -v /path/to/credentials.yaml:/credentials.yaml -p 8080:8080 ghcr.io/ente-io/ente-server
```
For more information on deployment you can check the [ente documentation](https://help.ente.io/self-hosting/)
## Pixelfed
[Pixelfed Technical Documentation > Configuration](https://docs.pixelfed.org/technical-documentation/env.html#filesystem)
## Pleroma
[Pleroma Documentation > Pleroma.Uploaders.S3](https://docs-develop.pleroma.social/backend/configuration/cheatsheet/#pleromauploaderss3)
### Creating your bucket
This is the usual Garage setup:
```bash
garage key new --name pleroma-key
garage bucket create pleroma
garage bucket allow pleroma --read --write --owner --key pleroma-key
```
We also need to expose these buckets publicly to serve their content to users:
```bash
garage bucket website --allow pleroma
```
Note the Key ID and Secret Key.
### Configure Pleroma
Update your Pleroma configuration like that in `/etc/pleroma/config.exs`.
```
config :pleroma, Pleroma.Upload,
uploader: Pleroma.Uploaders.S3,
base_url: "https://pleroma.garage.example.tld"
config :ex_aws, :s3,
access_key_id: "GW...",
secret_access_key: "XXX",
region: "garage",
host: "api.garage.example.tld"
```
And restart Pleroma.
You can found more information in [Pleroma Documentation > Pleroma.Uploaders.S3](https://docs-develop.pleroma.social/backend/configuration/cheatsheet/#pleromauploaderss3)
### Migrating your data
Pleroma have an internal migration tool that can encounter some fatal error
```
** (EXIT from #PID<0.98.0>) an exception was raised:
** (File.Error) could not stream "/var/lib/pleroma/uploads/09/f8": illegal operation on a directory
(elixir 1.17.3) lib/file/stream.ex:100: anonymous fn/3 in Enumerable.File.Stream.reduce/3
(elixir 1.17.3) lib/stream.ex:1675: anonymous fn/5 in Stream.resource/3
(elixir 1.17.3) lib/stream.ex:1891: Enumerable.Stream.do_each/4
(elixir 1.17.3) lib/task/supervised.ex:370: Task.Supervised.stream_reduce/7
(elixir 1.17.3) lib/enum.ex:4423: Enum.map/2
(ex_aws_s3 2.5.8) lib/ex_aws/s3/upload.ex:141: ExAws.Operation.ExAws.S3.Upload.perform/2
(pleroma 2.10.0) lib/pleroma/uploaders/s3.ex:60: Pleroma.Uploaders.S3.put_file/1
(pleroma 2.10.0) lib/pleroma/uploaders/uploader.ex:49: Pleroma.Uploaders.Uploader.put_file/2
```
So, use [your best tool](https://garagehq.deuxfleurs.fr/documentation/connect/cli/) to sync `/var/lib/pleroma/uploads/` in your S3.
Then, to avoid some non existent problem (just in case of), run this command
```bash
while true
do
rm -vr $(./bin/pleroma_ctl uploads migrate_local S3 2>&1 | grep "could not stream" | awk -F '"' '{print $2}')
sleep 5
done
```
If you have many files, stop this command sometime and the command bellow (interactive) to delete local
file after upload. Then restart the loop.
```bash
./bin/pleroma_ctl uploads migrate_local S3 --delete
```
And *voilà*
## Lemmy

View file

@ -207,3 +207,13 @@ $ plakar at @garageS3 ls
```
More information in Plakar documentation: https://www.plakar.io/docs/main/quickstart/
## Synology HyperBackup
HyperBackup can be configured to upload backups to garage using a custom S3 destination. However, the HyperBackup client hardcodes the `us-east-1` region that is a critical input to the v4 signature process. If garage is not set to `us-east-1`, HyperBackup will recognize available buckets, but fail during the final setup stage.
In garage.toml:
```toml
[s3_api]
s3_region = "us-east-1"
```

View file

@ -41,7 +41,7 @@ Some commands:
# list buckets
mc ls garage/
# list objets in a bucket
# list objects in a bucket
mc ls garage/my_files
# copy from your filesystem to garage
@ -149,6 +149,15 @@ rclone help
This will tremendously accelerate operations such as `rclone sync` or `rclone ncdu` by reducing the number
of ListObjects calls that are made.
**Garage behind Cloudflare proxy:** when running Garage behind Cloudflare proxy, you might see `Response: error 403 Forbidden, Forbidden: Invalid signature` error in your garage logs or `AccessDenied: Forbidden: Invalid signature` error in rclone logs. Try adding `--s3-sign-accept-encoding=false` flag to your rclone command and see if the issue is resolved.
```bash
# this throws an error
rclone lsd garage:
# this should work
rclone lsd --s3-sign-accept-encoding=false garage:
```
## `s3cmd`
@ -209,7 +218,7 @@ Within Cyberduck, a
available within the `Preferences -> Profiles` section. This can enabled and
then connections to Garage may be configured.
### Instuctions for the CLI
### Instructions for the CLI
To configure duck (Cyberduck's CLI tool), start by creating its folder hierarchy:
@ -314,4 +323,3 @@ ls
```
And through the web interface at http://[::1]:8080/web/client

View file

@ -201,11 +201,9 @@ on the binary cache, the client will download the result from the cache instead
### Channels
Channels additionnaly serve Nix definitions, ie. a `.nix` file referencing
Channels additionally serve Nix definitions, ie. a `.nix` file referencing
all the derivations you want to serve.
## Gitlab
*External link:* [Gitlab Documentation > Object storage](https://docs.gitlab.com/ee/administration/object_storage.html)

View file

@ -8,12 +8,12 @@ have published Ansible roles. We list them and compare them below.
## Comparison of Ansible roles
| Feature | [ansible-role-garage](#zorun-ansible-role-garage) | [garage-docker-ansible-deploy](#moan0s-garage-docker-ansible-deploy) | [eddster ansible-role-garage](#eddster-ansible-role-garage) |
| Feature | [ansible-role-garage](#zorun-ansible-role-garage) | [garage-docker-ansible-deploy](#moan0s-garage-docker-ansible-deploy) | [eddster2309 ansible-role-garage](#eddster2309-ansible-role-garage) |
|------------------------------------|---------------------------------------------|---------------------------------------------------------------|---------------------------------|
| **Runtime** | Systemd | Docker | Systemd |
| **Target OS** | Any Linux | Any Linux | Any Linux |
| **Architecture** | amd64, arm64, i686 | amd64, arm64 | arm64, arm, 386, amd64 |
| **Additional software** | None | Traefik | Ngnix and Keepalived (optional) |
| **Additional software** | None | Traefik | Nginx and Keepalived (optional) |
| **Automatic node connection** | ❌ | ✅ | ✅ |
| **Layout management** | ❌ | ✅ | ✅ |
| **Manage buckets & keys** | ❌ | ✅ (basic) | ✅ |

View file

@ -29,6 +29,10 @@ it's stable).
Garage is available in the official repositories under [extra](https://archlinux.org/packages/extra/x86_64/garage).
```bash
pacman -S garage
```
## FreeBSD
```bash
@ -40,3 +44,9 @@ pkg install garage
```bash
nix-shell -p garage
```
## conda-forge
```bash
pixi global install garage
```

View file

@ -33,7 +33,7 @@ by adding encryption at different levels.
We would be very curious to know your needs and thougs about ideas such as
encryption practices and things like key management, as we want Garage to be a
serious base platform for the developpment of secure, encrypted applications.
serious base platform for the development of secure, encrypted applications.
Do not hesitate to come talk to us if you have any thoughts or questions on the
subject.
@ -59,7 +59,7 @@ For standard S3 API requests, Garage does not encrypt data at rest by itself.
For the most generic at rest encryption of data, we recommend setting up your
storage partitions on encrypted LUKS devices.
If you are developping your own client software that makes use of S3 storage,
If you are developing your own client software that makes use of S3 storage,
we recommend implementing data encryption directly on the client side and never
transmitting plaintext data to Garage. This makes it easy to use an external
untrusted storage provider if necessary.
@ -108,14 +108,14 @@ Protects against the following threats:
- Stolen HDD
Crucially, does not protect againt malicious sysadmins or remote attackers that
Crucially, does not protect against malicious sysadmins or remote attackers that
might gain access to your servers.
Methods include full-disk encryption with tools such as LUKS.
## Encrypting data on the client side
Protects againt the following threats:
Protects against the following threats:
- A honest-but-curious administrator
- A malicious administrator that tries to corrupt your data

Some files were not shown because too many files have changed in this diff Show more