mirror of
https://git.deuxfleurs.fr/Deuxfleurs/garage.git
synced 2026-11-09 07:39:18 -05:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
99923a8718 | ||
|
0d0068317d | ||
|
b6b18427a5 | ||
|
9987166b2b | ||
|
|
b72b090a09 | ||
|
8551aefed4 | ||
|
|
47bf5d9fb0 | ||
|
|
5df37dae5e | ||
|
|
44af0bdab3 | ||
|
a7d6620e18 | ||
|
8eb12755e4 |
390 changed files with 14113 additions and 33121 deletions
|
|
@ -2,14 +2,13 @@ labels:
|
|||
nix: "enabled"
|
||||
|
||||
when:
|
||||
- event:
|
||||
- tag
|
||||
- pull_request
|
||||
- deployment
|
||||
- cron
|
||||
- manual
|
||||
- event: push
|
||||
branch: main-*
|
||||
event:
|
||||
- push
|
||||
- tag
|
||||
- pull_request
|
||||
- deployment
|
||||
- cron
|
||||
- manual
|
||||
|
||||
steps:
|
||||
- name: check formatting
|
||||
|
|
@ -17,16 +16,6 @@ steps:
|
|||
commands:
|
||||
- nix-build -j4 --attr flakePackages.fmt
|
||||
|
||||
- name: check typos
|
||||
image: nixpkgs/nix:nixos-24.05
|
||||
commands:
|
||||
- nix-shell --attr ci --run typos
|
||||
|
||||
- name: check lints with clippy
|
||||
image: nixpkgs/nix:nixos-24.05
|
||||
commands:
|
||||
- nix-build -j4 --attr flakePackages.clippy
|
||||
|
||||
- name: build
|
||||
image: nixpkgs/nix:nixos-24.05
|
||||
commands:
|
||||
|
|
@ -53,15 +42,3 @@ steps:
|
|||
- nix-build -j4 --attr flakePackages.dev
|
||||
- nix-shell --attr ci --run ./script/test-smoke.sh || (cat /tmp/garage.log; false)
|
||||
depends_on: [ build ]
|
||||
|
||||
- name: helm chart tests
|
||||
image: helmunittest/helm-unittest:4.2.3-1.1.2
|
||||
commands:
|
||||
- helm lint --strict script/helm/garage
|
||||
- helm lint --strict script/helm/garage -f script/helm/garage/tests/values/daemonset.yaml
|
||||
- helm lint --strict script/helm/garage -f script/helm/garage/tests/values/ingress.yaml
|
||||
- helm lint --strict script/helm/garage -f script/helm/garage/tests/values/existing-secret.yaml
|
||||
- helm lint --strict script/helm/garage -f script/helm/garage/tests/values/monitoring.yaml
|
||||
- helm lint --strict script/helm/garage -f script/helm/garage/tests/values/minimal.yaml
|
||||
- helm lint --strict script/helm/garage -f script/helm/garage/complex-values.yaml
|
||||
- helm unittest --strict script/helm/garage
|
||||
|
|
|
|||
|
|
@ -38,15 +38,7 @@ steps:
|
|||
- matrix:
|
||||
ARCH: i386
|
||||
|
||||
- name: upgrade tests from v1.0.0
|
||||
image: nixpkgs/nix:nixos-24.05
|
||||
commands:
|
||||
- nix-shell --attr ci --run "./script/test-upgrade.sh v1.0.0 x86_64-unknown-linux-musl" || (cat /tmp/garage.log; false)
|
||||
when:
|
||||
- matrix:
|
||||
ARCH: amd64
|
||||
|
||||
- name: upgrade tests from v0.8.4
|
||||
- name: upgrade tests
|
||||
image: nixpkgs/nix:nixos-24.05
|
||||
commands:
|
||||
- nix-shell --attr ci --run "./script/test-upgrade.sh v0.8.4 x86_64-unknown-linux-musl" || (cat /tmp/garage.log; false)
|
||||
|
|
|
|||
231
CONTRIBUTING.md
231
CONTRIBUTING.md
|
|
@ -1,231 +0,0 @@
|
|||
# Contributing to Garage
|
||||
|
||||
## Policy on AI
|
||||
|
||||
To ensure the quality of the codebase and documentation, the use of AI,
|
||||
including LLMs and coding agents, is strictly restricted in the following way:
|
||||
|
||||
- AI **must not** be used to write documentation
|
||||
|
||||
- **Do not** use AI to write bug reports, commit descriptions and pull request
|
||||
messages
|
||||
|
||||
- **Do not** use AI agents to make contributions to Garage, all contributions
|
||||
must be led by a human that know what they are doing at all times
|
||||
|
||||
- AI **may** be used for some tedious code generation tasks, limited to very
|
||||
mechanical translations from API docs or boilerplate writing. The code
|
||||
generated must be so simple as to make it clear that it cannot be covered by
|
||||
copyright.
|
||||
|
||||
You are free to make use of AI privately to explore the codebase and solve
|
||||
conceptual problems, but please restrain from copying the output from an LLM
|
||||
anywhere in your code or on the issue tracker, or from letting an agent edit
|
||||
the codebase directly.
|
||||
|
||||
|
||||
## Asking questions
|
||||
|
||||
Read the documentation before asking questions.
|
||||
Do not use the issue tracker to ask questions about Garage.
|
||||
Questions asked on the issue tracker will be closed.
|
||||
|
||||
Ask questions on the Matrix channel `#garage:deuxfleurs.fr` so that any
|
||||
community member can see your question and help you out.
|
||||
|
||||
If you need in-depth support from the Garage developers specifically, write to
|
||||
`garagehq@deuxfleurs.fr`. Even if you do so, we do not commit to giving you an
|
||||
answer.
|
||||
|
||||
|
||||
## Reporting bugs
|
||||
|
||||
When writing a bug report, use this checklist:
|
||||
|
||||
- For bugs that can be reproduced:
|
||||
- confirm that you are using the latest version of Garage and that the bug still exists in this version
|
||||
- set the log level to debug using the `RUST_LOG=garage=debug` environment variable and reproduce the bug to get more verbose logs
|
||||
|
||||
- Check whether there is already an open issue in the bug tracker. If so, your bug report is still valuable but please add it as a comment to the existing issue instead of opening a new one.
|
||||
|
||||
- Collect as much information as possible:
|
||||
- logs of the Garage daemon at the time the issue happened, including logs that show what was happening before the issue occurred
|
||||
- the output of `garage status`
|
||||
- the output of `garage stats -a`
|
||||
- the output of `garage layout history`
|
||||
|
||||
- Write a detailed bug report, including:
|
||||
- a description of your cluster (number of nodes, hardware, operating system, networking, etc)
|
||||
- a detailed description of what you did that led to the issue, including any code or command line that invoked a Garage API
|
||||
- what you were expecting
|
||||
- what actually happened, and how that's different from what you expected
|
||||
- the information collected previously
|
||||
- if possible, simple steps to help the developers reproduce the issue locally
|
||||
|
||||
Bug reports that are imprecise or otherwise unactionable will be closed.
|
||||
|
||||
|
||||
## Suggesting new features
|
||||
|
||||
Garage can be improved in many ways, but just suggesting a new feature does not mean we will implement it.
|
||||
Feature requests that may lead to an actual implementation are feature requests that:
|
||||
|
||||
- are precise and actionable, i.e. include a precise description of the expected behavior and any necessary architectural details required for the implementation
|
||||
- are motivated by actual need from a variety of users
|
||||
|
||||
Moreover, a certain number of features are defined as out-of-scope for Garage, including but not limited to:
|
||||
|
||||
- extensions to the S3 API that are not present on AWS
|
||||
- features that require the implementation of a consensus algorithm
|
||||
- more generally, features that are incompatible with the architecture of Garage and its goal of staying simple
|
||||
|
||||
Only feature requests in one of the following category may stay open in the issue tracker:
|
||||
|
||||
- features that the Garage team wants to work on
|
||||
- features that are being actively worked on by an external contributor which is clearly identified
|
||||
- features that are easy to implement and could be an easy task for a new contributor that wants to get to know the codebase
|
||||
|
||||
All other feature requests will be closed after a few months of inactivity, so as to keep the number of open issues to a manageable level.
|
||||
Feature requests that are clearly out of scope will be closed directly.
|
||||
|
||||
|
||||
## Improving the documentation
|
||||
|
||||
An easy way to contribute to Garage which also adds a lot of value is to
|
||||
improve the documentation. Make sure to write in clear technical English, and
|
||||
write unambiguously. Documentation contributions are very appreciated if they
|
||||
are well-written.
|
||||
|
||||
|
||||
## For developers
|
||||
|
||||
We welcome code contributions to Garage that adhere to our standards for quality:
|
||||
|
||||
- Changes should be reviewed from a functional perspective to ensure that they work well with the existing codebase and do not introduce bugs or subtle issues.
|
||||
|
||||
- You must have tested your contribution to make sure that it does what it says. The amount of testing required is proportional to the complexity of the change introduced.
|
||||
|
||||
- Any new feature must be properly documented following existing practices (see below).
|
||||
|
||||
- Unit tests should be included when relevant.
|
||||
|
||||
- Contributions should pass basic lints for syntactic quality (`cargo fmt`, `cargo clippy`, `typos`).
|
||||
|
||||
- Contributions should pass our CI test suite.
|
||||
|
||||
- No user-facing breaking changes may be introduced between major releases.
|
||||
|
||||
- No internal data model change may be introduced between major releases, to
|
||||
ensure that Garage daemons with different minor/patch versions numbers can
|
||||
work together in a cluster. For major releases, a proper migration path
|
||||
should be implemented and tested thoroughly.
|
||||
|
||||
Please follow up on your work when changes are requested, to avoid stale PRs.
|
||||
Do not take it personally if a Garage developer pushes directly to your branch
|
||||
to modify your contribution, as this might be necessary to get it merged
|
||||
faster.
|
||||
|
||||
### Properly documenting your contribution
|
||||
|
||||
#### Configuration options
|
||||
|
||||
New configuration options should be documented in
|
||||
`doc/book/reference-manual/configuration.md`. The documentation for a
|
||||
configuration option should be exhaustive. For instance, for choice options all
|
||||
choices should be listed explicitly with a precise description of their
|
||||
meaning.
|
||||
|
||||
In terms of syntax, all configuration options should appear in three places:
|
||||
|
||||
- in the example at the top, with an example value
|
||||
- in the index of all configuration options which is sorted by alphabetical order
|
||||
- in its dedicated subsection with full reference text
|
||||
|
||||
#### CLI commands and command flags
|
||||
|
||||
CLI commands are self-documented using the doc commends in the codebase.
|
||||
Make sure to write clear and precise comments for all options you are adding.
|
||||
|
||||
#### S3 features
|
||||
|
||||
If you implement new S3 features, make sure to update the compatibility matrix in `doc/book/reference-manual/s3-compatibility.md`.
|
||||
|
||||
#### Admin API
|
||||
|
||||
The admin API has an OpenAPI specification that is automatically generated
|
||||
using Utoipa, from a description of each endpoint that is given in
|
||||
`src/api/admin/openapi.rs` and a description of data structure schemas in
|
||||
`src/api/admin/api.rs`. The code in `openapi.rs` is only used to generate the
|
||||
OpenAPI specification document and not for the actual implementation in Garage,
|
||||
whereas structures defined in `api.rs` are also used for the implementation of
|
||||
API calls. Make sure to write good doc comments for all of these items so that
|
||||
the OpenAPI specification will be precise and accurate.
|
||||
|
||||
An up-to-date version of the OpenAPI specification document should be kept in
|
||||
the repository in `doc/api/garage-admin-v2.json`. When you are making changes
|
||||
to the admin API, update this document with the following command:
|
||||
|
||||
```
|
||||
cargo run -- admin-api-schema > doc/api/garage-admin-v2.json
|
||||
```
|
||||
|
||||
|
||||
## Garage team organization
|
||||
|
||||
Alex (handle `lx`) is the lead developer and is responsible of ensuring the
|
||||
correctness of Garage and stability between version upgrades.
|
||||
|
||||
The other maintainers are Trinity (handle `trinity-1686a`) and Maximilien (handle `halfa`).
|
||||
|
||||
Maximilien is responsible for coordinating effort on the Kubernetes integration / Helm chart.
|
||||
|
||||
## Pull request merging criteria
|
||||
|
||||
The following PRs should only be merged after review and approval from Alex:
|
||||
|
||||
- PRs that introduce architectural changes, such as changes in the data model
|
||||
or change in the coordination protocols between nodes
|
||||
|
||||
- PRs that introduce changes on the format of data structures used for
|
||||
persistent disk storage and internal cluster communication (RPC)
|
||||
|
||||
- PRs that are suspected of introducing some kind of breakage or unexpected
|
||||
behavior due to their complexity
|
||||
|
||||
PRs that introduce breaking change for users but don't fall in one of the
|
||||
previous category should be discussed between maintainers to evaluate the
|
||||
impact on users when upgrading. Alex's approval is not required to merge them
|
||||
as long as they are clearly identified as breaking in the PR title, and are
|
||||
properly merged in the branch for the next major version and not in the current
|
||||
main branch.
|
||||
|
||||
All other PRs can be merged by any maintainer on their own, once they are
|
||||
confident that the quality standards defined in this document are respected
|
||||
before merging.
|
||||
|
||||
## Merging strategy
|
||||
|
||||
When merging PRs, maintainers should ensure that a Git commit is created by
|
||||
Forgejo that records the PR number, its title and its text in the commit
|
||||
message. If a PR is fixing an issue, make sure that the issue number is
|
||||
included in the PR title as well. This is to ensure that when releasing a new
|
||||
version of Garage, the changelog in the release notes can be properly
|
||||
constructed by reading the Git log since the last release.
|
||||
|
||||
We also want to keep the history "almost linear" to facilitate the use of `git
|
||||
bisect` if it ever were necessary. This leaves the following two merging
|
||||
strategies:
|
||||
|
||||
- For PRs that consist of many commits that should stay independent, the
|
||||
"rebase and create merge commit" strategy should be used. The merge commit is
|
||||
created automatically by Forgejo and saves the PR's number, title and text in
|
||||
the commit message.
|
||||
|
||||
- For PRs that consist of only one commit, or a few number of commits that can
|
||||
be merged, the "create squash commit" strategy should be used. This way a
|
||||
single commit will be created by Forgejo which also saves the PR's number,
|
||||
title and text in the commit message.
|
||||
|
||||
When cherry-picking commits from one branch to the other, a simple fast-forward
|
||||
merging strategy can be used if the commit message already references a PR
|
||||
number.
|
||||
3092
Cargo.lock
generated
3092
Cargo.lock
generated
File diff suppressed because it is too large
Load diff
186
Cargo.toml
186
Cargo.toml
|
|
@ -16,7 +16,6 @@ members = [
|
|||
"src/garage",
|
||||
"src/k2v-client",
|
||||
"src/format-table",
|
||||
"fuzz",
|
||||
]
|
||||
|
||||
default-members = ["src/garage"]
|
||||
|
|
@ -25,182 +24,131 @@ default-members = ["src/garage"]
|
|||
|
||||
# Internal Garage crates
|
||||
format_table = { version = "0.1.1", path = "src/format-table" }
|
||||
garage_api_common = { version = "2.5.0", path = "src/api/common" }
|
||||
garage_api_admin = { version = "2.5.0", path = "src/api/admin" }
|
||||
garage_api_s3 = { version = "2.5.0", path = "src/api/s3" }
|
||||
garage_api_k2v = { version = "2.5.0", path = "src/api/k2v" }
|
||||
garage_block = { version = "2.5.0", path = "src/block" }
|
||||
garage_db = { version = "2.5.0", path = "src/db", default-features = false }
|
||||
garage_model = { version = "2.5.0", path = "src/model", default-features = false }
|
||||
garage_net = { version = "2.5.0", path = "src/net" }
|
||||
garage_rpc = { version = "2.5.0", path = "src/rpc" }
|
||||
garage_table = { version = "2.5.0", path = "src/table" }
|
||||
garage_util = { version = "2.5.0", path = "src/util" }
|
||||
garage_web = { version = "2.5.0", path = "src/web" }
|
||||
garage_api_common = { version = "1.3.1", path = "src/api/common" }
|
||||
garage_api_admin = { version = "1.3.1", path = "src/api/admin" }
|
||||
garage_api_s3 = { version = "1.3.1", path = "src/api/s3" }
|
||||
garage_api_k2v = { version = "1.3.1", path = "src/api/k2v" }
|
||||
garage_block = { version = "1.3.1", path = "src/block" }
|
||||
garage_db = { version = "1.3.1", path = "src/db", default-features = false }
|
||||
garage_model = { version = "1.3.1", path = "src/model", default-features = false }
|
||||
garage_net = { version = "1.3.1", path = "src/net" }
|
||||
garage_rpc = { version = "1.3.1", path = "src/rpc" }
|
||||
garage_table = { version = "1.3.1", path = "src/table" }
|
||||
garage_util = { version = "1.3.1", path = "src/util" }
|
||||
garage_web = { version = "1.3.1", path = "src/web" }
|
||||
k2v-client = { version = "0.0.4", path = "src/k2v-client" }
|
||||
|
||||
# External crates from crates.io
|
||||
arc-swap = "1.8"
|
||||
arbitrary = { version = "1.4.2"}
|
||||
argon2 = { version = "0.5", features = ["std"] }
|
||||
async-trait = "0.1"
|
||||
arc-swap = "1.0"
|
||||
argon2 = "0.5"
|
||||
async-trait = "0.1.7"
|
||||
backtrace = "0.3"
|
||||
base64 = "0.22"
|
||||
base64 = "0.21"
|
||||
blake2 = "0.10"
|
||||
bytes = "1.11"
|
||||
bytesize = "2.3"
|
||||
bytes = "1.0"
|
||||
bytesize = "1.1"
|
||||
cfg-if = "1.0"
|
||||
chrono = { version = "0.4", features = ["serde"] }
|
||||
crc-fast = "1.9"
|
||||
chrono = "0.4"
|
||||
crc32fast = "1.4"
|
||||
crc32c = "0.6"
|
||||
crypto-common = "0.1"
|
||||
fundu = "2.0"
|
||||
fundu-systemd = "0.3"
|
||||
gethostname = "1.1"
|
||||
git-version = "0.3"
|
||||
gethostname = "0.4"
|
||||
git-version = "0.3.4"
|
||||
hex = "0.4"
|
||||
hexdump = "0.1"
|
||||
html-escape = "0.2.13"
|
||||
hmac = "0.12"
|
||||
itertools = "0.14"
|
||||
ipnet = "2.11"
|
||||
lazy_static = "1.5"
|
||||
libfuzzer-sys = "0.4"
|
||||
itertools = "0.12"
|
||||
ipnet = "2.9.0"
|
||||
lazy_static = "1.4"
|
||||
md-5 = "0.10"
|
||||
mktemp = "0.5"
|
||||
nix = { version = "0.31", default-features = false, features = ["fs"] }
|
||||
nom = "8.0"
|
||||
nix = { version = "0.29", default-features = false, features = ["fs"] }
|
||||
nom = "7.1"
|
||||
parking_lot = "0.12"
|
||||
paste = "1.0"
|
||||
pin-project = "1.1"
|
||||
pnet_datalink = "0.35"
|
||||
rand = "0.9"
|
||||
parse_duration = "2.1"
|
||||
pin-project = "1.0.12"
|
||||
pnet_datalink = "0.34"
|
||||
rand = "0.8"
|
||||
sha1 = "0.10"
|
||||
sha2 = "0.10"
|
||||
subtle = "2.6.1"
|
||||
timeago = { version = "0.5", default-features = false }
|
||||
timeago = { version = "0.4", default-features = false }
|
||||
xxhash-rust = { version = "0.8", default-features = false, features = ["xxh3"] }
|
||||
|
||||
aes-gcm = { version = "0.10", features = ["aes", "stream"] }
|
||||
sodiumoxide = { version = "0.2.5-0", package = "kuska-sodiumoxide" }
|
||||
kuska-handshake = { version = "0.2.0", features = ["default", "async_std"] }
|
||||
|
||||
clap = { version = "4.5", features = ["derive", "env"] }
|
||||
clap = { version = "4.1", features = ["derive", "env"] }
|
||||
pretty_env_logger = "0.5"
|
||||
structopt = { version = "0.3", default-features = false }
|
||||
syslog-tracing = "0.3"
|
||||
tracing = "0.1"
|
||||
tracing-journald = "0.3"
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
|
||||
tracing-journald = "0.3.1"
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|
||||
|
||||
heed = { version = "0.22", default-features = false, features = [] }
|
||||
rusqlite = { version = "0.38", features = ["fallible_uint"] }
|
||||
heed = { version = "0.11", default-features = false, features = ["lmdb"] }
|
||||
rusqlite = "0.37"
|
||||
r2d2 = "0.8"
|
||||
r2d2_sqlite = "0.32"
|
||||
fjall = "2.11"
|
||||
r2d2_sqlite = "0.31"
|
||||
fjall = "2.4"
|
||||
|
||||
async-compression = { version = "0.4", features = ["tokio", "zstd"] }
|
||||
zstd = { version = "0.13", default-features = false }
|
||||
|
||||
quick-xml = { version = "0.39", features = ["serialize"] }
|
||||
rmp-serde = "1.3"
|
||||
quick-xml = { version = "0.26", features = [ "serialize" ] }
|
||||
rmp-serde = "1.1.2"
|
||||
serde = { version = "1.0", default-features = false, features = ["derive", "rc"] }
|
||||
serde_bytes = "0.11"
|
||||
serde_json = "1.0"
|
||||
toml = { version = "0.9", default-features = false, features = ["parse", "serde"] }
|
||||
utoipa = { version = "5.4", features = ["chrono"] }
|
||||
toml = { version = "0.8", default-features = false, features = ["parse"] }
|
||||
|
||||
# newer version requires rust edition 2021
|
||||
k8s-openapi = { version = "0.27", features = ["v1_35"] }
|
||||
kube = { version = "3.0", default-features = false, features = [
|
||||
"runtime",
|
||||
"derive",
|
||||
"client",
|
||||
"rustls-tls",
|
||||
] }
|
||||
schemars = "1.2"
|
||||
reqwest = { version = "0.13", default-features = false, features = [
|
||||
"rustls-no-provider",
|
||||
"json",
|
||||
] }
|
||||
rustls = { version = "0.23", default-features = false, features = ["std", "ring"] }
|
||||
k8s-openapi = { version = "0.21", features = ["v1_24"] }
|
||||
kube = { version = "0.88", default-features = false, features = ["runtime", "derive", "client", "rustls-tls"] }
|
||||
schemars = "0.8"
|
||||
reqwest = { version = "0.11", default-features = false, features = ["rustls-tls-manual-roots", "json"] }
|
||||
|
||||
form_urlencoded = "1.2"
|
||||
http = "1.4"
|
||||
form_urlencoded = "1.0.0"
|
||||
http = "1.0"
|
||||
httpdate = "1.0"
|
||||
http-range = "0.1"
|
||||
http-body-util = "0.1"
|
||||
hyper = { version = "1.8", default-features = false }
|
||||
hyper-util = { version = "0.1", features = ["full"] }
|
||||
multer = "3.1"
|
||||
percent-encoding = "2.3"
|
||||
roxmltree = "0.21"
|
||||
url = "2.5"
|
||||
hyper = { version = "1.0", default-features = false }
|
||||
hyper-util = { version = "0.1", features = [ "full" ] }
|
||||
multer = "3.0"
|
||||
percent-encoding = "2.2"
|
||||
roxmltree = "0.19"
|
||||
url = "2.3"
|
||||
|
||||
futures = "0.3"
|
||||
futures-util = "0.3"
|
||||
tokio = { version = "1.49", default-features = false, features = [
|
||||
"rt",
|
||||
"rt-multi-thread",
|
||||
"io-util",
|
||||
"net",
|
||||
"time",
|
||||
"macros",
|
||||
"sync",
|
||||
"signal",
|
||||
"fs",
|
||||
] }
|
||||
tokio = { version = "1.0", default-features = false, features = ["net", "rt", "rt-multi-thread", "io-util", "net", "time", "macros", "sync", "signal", "fs"] }
|
||||
tokio-util = { version = "0.7", features = ["compat", "io"] }
|
||||
tokio-stream = { version = "0.1", features = ["net"] }
|
||||
socket2 = { version = "0.6", features = ["all"] }
|
||||
|
||||
opentelemetry = { version = "0.32", features = ["metrics", "trace"] }
|
||||
opentelemetry_sdk = "0.32"
|
||||
opentelemetry-prometheus = "0.32"
|
||||
opentelemetry-otlp = { version = "0.32", default-features = false, features = [
|
||||
"grpc-tonic",
|
||||
"trace",
|
||||
"internal-logs",
|
||||
] }
|
||||
opentelemetry-contrib = "0.24"
|
||||
prometheus = "0.14"
|
||||
opentelemetry = { version = "0.17", features = [ "rt-tokio", "metrics", "trace" ] }
|
||||
opentelemetry-prometheus = "0.10"
|
||||
opentelemetry-otlp = "0.10"
|
||||
opentelemetry-contrib = "0.9"
|
||||
prometheus = "0.13"
|
||||
|
||||
# used by the k2v-client crate only
|
||||
aws-sigv4 = { version = "1.3", default-features = false }
|
||||
hyper-rustls = { version = "0.27", default-features = false, features = [
|
||||
"http1",
|
||||
"http2",
|
||||
"ring",
|
||||
"rustls-native-certs",
|
||||
] }
|
||||
aws-sigv4 = { version = "1.1", default-features = false }
|
||||
hyper-rustls = { version = "0.26", default-features = false, features = ["http1", "http2", "ring", "rustls-native-certs"] }
|
||||
log = "0.4"
|
||||
thiserror = "2.0"
|
||||
|
||||
# ---- used only as build / dev dependencies ----
|
||||
assert-json-diff = "2.0"
|
||||
rustc_version = "0.4"
|
||||
rustc_version = "0.4.0"
|
||||
static_init = "1.0"
|
||||
aws-smithy-runtime = { version = "1.9", default-features = false, features = [
|
||||
"tls-rustls",
|
||||
] }
|
||||
aws-sdk-config = { version = "1.99", default-features = false }
|
||||
aws-sdk-s3 = { version = "1.121", default-features = false, features = [
|
||||
"rt-tokio",
|
||||
] }
|
||||
aws-smithy-runtime = { version = "1.8", default-features = false, features = ["tls-rustls"] }
|
||||
aws-sdk-config = { version = "1.62", default-features = false }
|
||||
aws-sdk-s3 = { version = "1.79", default-features = false, features = ["rt-tokio"] }
|
||||
|
||||
[profile.release]
|
||||
lto = "thin"
|
||||
codegen-units = 16
|
||||
opt-level = 3
|
||||
strip = "debuginfo"
|
||||
|
||||
[workspace.lints.clippy]
|
||||
# pedantic lints configuration
|
||||
doc_markdown = "warn"
|
||||
format_collect = "warn"
|
||||
manual_midpoint = "warn"
|
||||
semicolon_if_nothing_returned = "warn"
|
||||
unnecessary_semicolon = "warn"
|
||||
unnecessary_wraps = "warn"
|
||||
|
||||
# nursery lints configuration
|
||||
# or_fun_call = "warn" # enable it to help detect non trivial code used in `_or` method
|
||||
|
|
|
|||
|
|
@ -1,44 +0,0 @@
|
|||
# Governance of Garage
|
||||
|
||||
This documents how the Garage project operates. It reflects the state of the project as of July 2026 and is not optimal. The team is interested to improve it in the future.
|
||||
|
||||
## Team organization
|
||||
|
||||
* **Contributors**: anyone can contribute by proposing changes in issues and pull requests.
|
||||
|
||||
* **Maintainers**: they are responsible for reviewing, merging pull requests, publishing releases and triaging issues.
|
||||
The current maintainers are:
|
||||
* Alex (handle `lx`)
|
||||
* Trinity (handle `trinity-1686a`)
|
||||
* Maximilien (handle `halfa`), who is in particular responsible for coordinating effort on the Kubernetes integration / Helm chart.
|
||||
|
||||
They are added to a white-list of the branch protection rule of the repository to enable them to merge pull requests.
|
||||
To become a maintainer, you need to be a long-term contributor and earn the personal trust of Alex.
|
||||
There is no set process for leaving the maintainer role.
|
||||
|
||||
* **Lead developer**: Alex (handle `lx`) is the lead developer and is responsible of ensuring the
|
||||
correctness of Garage and stability between version upgrades. He may transfer this role to someone else as he sees fit.
|
||||
|
||||
## Communication channels
|
||||
|
||||
The team coordinates in the following channels:
|
||||
* The issue tracker and pull requests of the official repository.
|
||||
* The `#garage:deuxfleurs.fr` matrix channel (in English), open to anyone.
|
||||
On this channel, users may ask for support and discussions about development also happen.
|
||||
* The `#garage-dev:deuxfleurs.fr` matrix channel (in French), not advertised to contributors but de facto accessible to anyone.
|
||||
Discussions about development and project coordination happen there.
|
||||
|
||||
The moderators for those discussion channels are the Garage maintainers.
|
||||
|
||||
## Decision procedures
|
||||
|
||||
Decisions are taken by lazy consensus, with the lead developer settling discussions when a consensus cannot be reached.
|
||||
|
||||
## Governance changes
|
||||
|
||||
There is no set process for changing the governance of garage.
|
||||
|
||||
## See also
|
||||
|
||||
* [Project goals](https://garagehq.deuxfleurs.fr/documentation/design/goals/)
|
||||
* [Contributing instructions](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main-v2/CONTRIBUTING.md)
|
||||
5
Makefile
5
Makefile
|
|
@ -1,4 +1,4 @@
|
|||
.PHONY: doc all run1 run2 run3 helmtest
|
||||
.PHONY: doc all run1 run2 run3
|
||||
|
||||
all:
|
||||
clear
|
||||
|
|
@ -20,6 +20,3 @@ run3:
|
|||
RUST_LOG=garage=debug ./target/debug/garage -c tmp/config3.toml server
|
||||
run3rel:
|
||||
RUST_LOG=garage=debug ./target/release/garage -c tmp/config3.toml server
|
||||
|
||||
helmtest:
|
||||
helm unittest --strict script/helm/garage
|
||||
|
|
|
|||
14
SECURITY.md
14
SECURITY.md
|
|
@ -1,14 +0,0 @@
|
|||
# Security Reporting
|
||||
|
||||
If you wish to report responsibly a security vulnerability about Garage, we ask that you follow the following process.
|
||||
|
||||
Please report each security vulnerabilities by filling out the following template:
|
||||
|
||||
- PROJECT: A URL to the code repository containing the vulnerable version - be reminded that the source of truth is at https://git.deuxfleurs.fr/deuxfleurs/garage
|
||||
- PUBLIC: Please let us know if this vulnerability has been made or discussed publicly already, and if so, please let us know where.
|
||||
- DESCRIPTION: Please provide precise description of the security vulnerability you have found with as much information as you are able and willing to provide.
|
||||
|
||||
Please send the above info, along with any other information you feel is pertinent by emailing the core team at: garagehq@deuxfleurs.fr
|
||||
|
||||
The Garage Core Team will let you know within a few weeks whether or not your report has been accepted or rejected.
|
||||
We ask that you please keep the report confidential until we have either responded or made a public announcement.
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Garage administration API v0</title>
|
||||
<title>Garage Adminstration API v0</title>
|
||||
<!-- needed for adaptive design -->
|
||||
<meta charset="utf-8"/>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
|
|
|
|||
|
|
@ -3,9 +3,9 @@ info:
|
|||
version: v0.8.0
|
||||
title: Garage Administration API v0+garage-v0.8.0
|
||||
description: |
|
||||
Administrate your Garage cluster programmatically, including status, layout, keys, buckets, and maintenance tasks.
|
||||
Administrate your Garage cluster programatically, including status, layout, keys, buckets, and maintainance tasks.
|
||||
|
||||
*Disclaimer: The API is not stable yet, hence its v0 tag. The API can change at any time, and changes can include breaking backward compatibility. Read the changelog and upgrade your scripts before upgrading. Additionally, this specification is very early stage and can contain bugs, especially on error return codes/types that are not tested yet. Do not expect a well finished and polished product!*
|
||||
*Disclaimer: The API is not stable yet, hence its v0 tag. The API can change at any time, and changes can include breaking backward compatibility. Read the changelog and upgrade your scripts before upgrading. Additionnaly, this specification is very early stage and can contain bugs, especially on error return codes/types that are not tested yet. Do not expect a well finished and polished product!*
|
||||
paths:
|
||||
/status:
|
||||
get:
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Garage administration API v1</title>
|
||||
<title>Garage Adminstration API v0</title>
|
||||
<!-- needed for adaptive design -->
|
||||
<meta charset="utf-8"/>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
|
|
|
|||
|
|
@ -3,9 +3,9 @@ info:
|
|||
version: v0.9.0
|
||||
title: Garage Administration API v0+garage-v0.9.0
|
||||
description: |
|
||||
Administrate your Garage cluster programmatically, including status, layout, keys, buckets, and maintenance tasks.
|
||||
Administrate your Garage cluster programatically, including status, layout, keys, buckets, and maintainance tasks.
|
||||
|
||||
*Disclaimer: The API is not stable yet, hence its v0 tag. The API can change at any time, and changes can include breaking backward compatibility. Read the changelog and upgrade your scripts before upgrading. Additionally, this specification is very early stage and can contain bugs, especially on error return codes/types that are not tested yet. Do not expect a well finished and polished product!*
|
||||
*Disclaimer: The API is not stable yet, hence its v0 tag. The API can change at any time, and changes can include breaking backward compatibility. Read the changelog and upgrade your scripts before upgrading. Additionnaly, this specification is very early stage and can contain bugs, especially on error return codes/types that are not tested yet. Do not expect a well finished and polished product!*
|
||||
paths:
|
||||
/health:
|
||||
get:
|
||||
|
|
@ -440,7 +440,7 @@ paths:
|
|||
- "false"
|
||||
example: "true"
|
||||
required: false
|
||||
description: "Whether or not the secret key should be returned in the response"
|
||||
description: "Wether or not the secret key should be returned in the response"
|
||||
responses:
|
||||
'500':
|
||||
description: "The server can not handle your request. Check your connectivity with the rest of the cluster."
|
||||
|
|
|
|||
|
|
@ -1,24 +0,0 @@
|
|||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Garage administration API v2</title>
|
||||
<!-- needed for adaptive design -->
|
||||
<meta charset="utf-8"/>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<link href="./css/redoc.css" rel="stylesheet">
|
||||
|
||||
<!--
|
||||
Redoc doesn't change outer page styles
|
||||
-->
|
||||
<style>
|
||||
body {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<redoc spec-url='./garage-admin-v2.json'></redoc>
|
||||
<script src="./redoc.standalone.js"> </script>
|
||||
</body>
|
||||
</html>
|
||||
File diff suppressed because it is too large
Load diff
File diff suppressed because one or more lines are too long
|
|
@ -51,4 +51,4 @@ We are currently building this SDK for [Python](@/documentation/build/python.md#
|
|||
|
||||
More information:
|
||||
- [In the reference manual](@/documentation/reference-manual/admin-api.md)
|
||||
- [Full specification](https://garagehq.deuxfleurs.fr/api/garage-admin-v0.html)
|
||||
- [Full specifiction](https://garagehq.deuxfleurs.fr/api/garage-admin-v0.html)
|
||||
|
|
|
|||
|
|
@ -5,13 +5,13 @@ weight = 99
|
|||
|
||||
## S3
|
||||
|
||||
If you are developing a new application, you may want to use Garage to store your user's media.
|
||||
If you are developping a new application, you may want to use Garage to store your user's media.
|
||||
|
||||
The S3 API that Garage uses is a standard REST API, so as long as you can make HTTP requests,
|
||||
you can query it. You can check the [S3 REST API Reference](https://docs.aws.amazon.com/AmazonS3/latest/API/API_Operations_Amazon_Simple_Storage_Service.html) from Amazon to learn more.
|
||||
|
||||
Developing your own wrapper around the REST API is time consuming and complicated.
|
||||
Instead, there are some libraries already available.
|
||||
Developping your own wrapper around the REST API is time consuming and complicated.
|
||||
Instead, there are some libraries already avalaible.
|
||||
|
||||
Some of them are maintained by Amazon, some by Minio, others by the community.
|
||||
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ To configure S3-compatible software to interact with Garage,
|
|||
you will need the following parameters:
|
||||
|
||||
- An **API endpoint**: this corresponds to the HTTP or HTTPS address
|
||||
used to contact the Garage server. When running Garage locally this will usually
|
||||
used to contact the Garage server. When runing Garage locally this will usually
|
||||
be `http://127.0.0.1:3900`. In a real-world setting, you would usually have a reverse-proxy
|
||||
that adds TLS support and makes your Garage server available under a public hostname
|
||||
such as `https://garage.example.com`.
|
||||
|
|
|
|||
|
|
@ -12,9 +12,8 @@ In this section, we cover the following web applications:
|
|||
| [Mastodon](#mastodon) | ✅ | Natively supported |
|
||||
| [Matrix](#matrix) | ✅ | Tested with `synapse-s3-storage-provider` |
|
||||
| [ejabberd](#ejabberd) | ✅ | `mod_s3_upload` |
|
||||
| [Ente](#ente) | ✅ | Natively supported |
|
||||
| [Pixelfed](#pixelfed) | ❓ | Natively supported |
|
||||
| [Pleroma](#pleroma) | ✅ | Natively supported |
|
||||
| [Pixelfed](#pixelfed) | ✅ | Natively supported |
|
||||
| [Pleroma](#pleroma) | ❓ | Not yet tested |
|
||||
| [Lemmy](#lemmy) | ✅ | Supported with pict-rs |
|
||||
| [Funkwhale](#funkwhale) | ❓ | Not yet tested |
|
||||
| [Misskey](#misskey) | ❓ | Not yet tested |
|
||||
|
|
@ -54,7 +53,7 @@ garage bucket allow nextcloud --read --write --key nextcloud-key
|
|||
|
||||
Now edit your Nextcloud configuration file to enable object storage.
|
||||
On my installation, the config. file is located at the following path: `/var/www/nextcloud/config/config.php`.
|
||||
We will add a new root key to the `$CONFIG` dictionary named `objectstore`:
|
||||
We will add a new root key to the `$CONFIG` dictionnary named `objectstore`:
|
||||
|
||||
```php
|
||||
<?php
|
||||
|
|
@ -413,7 +412,7 @@ mc mirror --newer-than "3h" ./public/system/ garage/mastodon-data
|
|||
|
||||
## Matrix
|
||||
|
||||
Matrix is a chat communication protocol. Its main stable server implementation, [Synapse](https://matrix-org.github.io/synapse/latest/), provides a module to store media on a S3 backend. Additionally, a server independent media store supporting S3 has been developed by the community, it has been made possible thanks to how the matrix API has been designed and will work with implementations like Conduit, Dendrite, etc.
|
||||
Matrix is a chat communication protocol. Its main stable server implementation, [Synapse](https://matrix-org.github.io/synapse/latest/), provides a module to store media on a S3 backend. Additionally, a server independent media store supporting S3 has been developped by the community, it has been made possible thanks to how the matrix API has been designed and will work with implementations like Conduit, Dendrite, etc.
|
||||
|
||||
### synapse-s3-storage-provider (synapse only)
|
||||
|
||||
|
|
@ -450,7 +449,7 @@ media_storage_providers:
|
|||
|
||||
Note that uploaded media will also be stored locally and this behavior can not be deactivated, it is even required for
|
||||
some operations like resizing images.
|
||||
In fact, your local filesystem is considered as a cache but without any automated way to garbage collect it.
|
||||
In fact, your local filesysem is considered as a cache but without any automated way to garbage collect it.
|
||||
|
||||
We can build our garbage collector with `s3_media_upload`, a tool provided with the module.
|
||||
If you installed the module with the command provided before, you should be able to bring it in your path:
|
||||
|
|
@ -547,7 +546,7 @@ ejabberdctl module_install mod_s3_upload
|
|||
Create the required key and bucket with:
|
||||
|
||||
```bash
|
||||
garage key create ejabberd
|
||||
garage key new --name ejabberd
|
||||
garage bucket create objects.xmpp-server.fr
|
||||
garage bucket allow objects.xmpp-server.fr --read --write --key ejabberd
|
||||
garage bucket website --allow objects.xmpp-server.fr
|
||||
|
|
@ -568,186 +567,13 @@ The module can then be configured with:
|
|||
Other configuration options can be found in the
|
||||
[configuration YAML file](https://github.com/processone/ejabberd-contrib/blob/master/mod_s3_upload/conf/mod_s3_upload.yml).
|
||||
|
||||
|
||||
## Ente
|
||||
|
||||
Ente is an alternative for Google Photos and Apple Photos. It [can be selfhosted](https://help.ente.io/self-hosting/) and is working fine with Garage as of May 2024.
|
||||
As a first step we need to create a bucket and a key for Ente:
|
||||
|
||||
```bash
|
||||
garage bucket create ente
|
||||
garage key create ente-key
|
||||
# For the CORS setup to work, the key needs to be --owner as well, at least temporarily.
|
||||
garage bucket allow ente --read --write --owner --key ente-key
|
||||
```
|
||||
|
||||
We also need to setup some CORS rules to allow the Ente frontend to access the bucket:
|
||||
|
||||
```bash
|
||||
export CORS='{"CORSRules":[{"AllowedHeaders":["*"],"AllowedMethods":["GET", "PUT", "POST", "DELETE"],"AllowedOrigins":["*"], "ExposeHeaders":["ETag"]}]}'
|
||||
aws s3api put-bucket-cors --bucket ente --cors-configuration $CORS
|
||||
```
|
||||
|
||||
Now we need to configure ente-server to use our bucket. This is explained [in the Ente S3 documentation](https://help.ente.io/self-hosting/guides/external-s3).
|
||||
Prepare a configuration file for ente's backend as `museum.yaml`:
|
||||
|
||||
```yaml
|
||||
credentials-file: /credentials.yaml
|
||||
apps:
|
||||
public-albums: https://albums.example.tld # If you want to use the share album feature
|
||||
internal:
|
||||
hardcoded-ott:
|
||||
local-domain-suffix: "@example.com" # Your domain
|
||||
local-domain-value: 123456 # Custom One-Time Password since we are not sending mail by default
|
||||
key:
|
||||
# WARNING -- You MUST CHANGE the values below
|
||||
# Someone has made an image that can do it for you : https://github.com/EdyTheCow/ente-selfhost/blob/main/images/ente-server-tools/Dockerfile
|
||||
# Simply build it yourself or run docker run --rm ghcr.io/edythecow/ente-server-tools go run tools/gen-random-keys/main.go
|
||||
encryption: yvmG/RnzKrbCb9L3mgsmoxXr9H7i2Z4qlbT0mL3ln4w= # CHANGE THIS VALUE
|
||||
hash: KXYiG07wC7GIgvCSdg+WmyWdXDAn6XKYJtp/wkEU7x573+byBRAYtpTP0wwvi8i/4l37uicX1dVTUzwH3sLZyw== # CHANGE THIS VALUE
|
||||
jwt:
|
||||
secret: i2DecQmfGreG6q1vBj5tCokhlN41gcfS2cjOs9Po-u8= # CHANGE THIS VALUE
|
||||
```
|
||||
|
||||
The full configuration file can be found [here](https://github.com/ente-io/ente/blob/main/server/configurations/local.yaml)
|
||||
Then prepare a credentials file as `credentials.yaml`
|
||||
|
||||
```yaml
|
||||
db:
|
||||
host: postgres
|
||||
port: 5432
|
||||
name: <ente_db_name>
|
||||
user: <pguser>
|
||||
password: <pgpass>
|
||||
|
||||
s3:
|
||||
# Override the primary and secondary hot storage. The commented out values
|
||||
# are the defaults.
|
||||
#
|
||||
hot_storage:
|
||||
primary: b2-eu-cen
|
||||
# secondary: wasabi-eu-central-2-v3
|
||||
|
||||
# If true, enable some workarounds to allow us to use a local minio instance
|
||||
# for object storage.
|
||||
#
|
||||
# 1. Disable SSL.
|
||||
# 2. Use "path" style S3 URLs (see `use_path_style_urls` below).
|
||||
# 3. Directly download the file during replication instead of going via the
|
||||
# Cloudflare worker.
|
||||
# 4. Do not specify storage classes when uploading objects (since minio does
|
||||
# not support them, specifically it doesn't support GLACIER).
|
||||
are_local_buckets: true
|
||||
|
||||
# To use "path" style S3 URLs instead of DNS-based bucket access
|
||||
# default to true if you set "are_local_buckets: true"
|
||||
# use_path_style_urls: true
|
||||
|
||||
b2-eu-cen: # Don't change this key, it is hardcoded
|
||||
key: <keyID>
|
||||
secret: <keySecret>
|
||||
endpoint: garage:3900 # publicly accessible endpoint of your garage instance
|
||||
region: garage
|
||||
bucket: <yourbucketName>
|
||||
use_path_style: true
|
||||
# you can specify secondary locations, names are hardcoded as well
|
||||
# wasabi-eu-central-2-v3:
|
||||
# scw-eu-fr-v3:
|
||||
|
||||
# and you can also specify a bucket to be used for embeddings, preview etc..
|
||||
# default to the first bucket
|
||||
# derived-storage: wasabi-eu-central-2-derived
|
||||
```
|
||||
|
||||
Finally you can run it with Docker :
|
||||
|
||||
```bash
|
||||
docker run -d --name ente-server --restart unless-stopped -v /path/to/museum.yaml:/museum.yaml -v /path/to/credentials.yaml:/credentials.yaml -p 8080:8080 ghcr.io/ente-io/ente-server
|
||||
```
|
||||
|
||||
For more information on deployment you can check the [ente documentation](https://help.ente.io/self-hosting/)
|
||||
|
||||
## Pixelfed
|
||||
|
||||
[Pixelfed Technical Documentation > Configuration](https://docs.pixelfed.org/technical-documentation/env.html#filesystem)
|
||||
|
||||
## Pleroma
|
||||
|
||||
### Creating your bucket
|
||||
|
||||
This is the usual Garage setup:
|
||||
|
||||
```bash
|
||||
garage key create pleroma-key
|
||||
garage bucket create pleroma
|
||||
garage bucket allow pleroma --read --write --owner --key pleroma-key
|
||||
```
|
||||
|
||||
We also need to expose these buckets publicly to serve their content to users:
|
||||
|
||||
```bash
|
||||
garage bucket website --allow pleroma
|
||||
```
|
||||
|
||||
Note the Key ID and Secret Key.
|
||||
|
||||
### Configure Pleroma
|
||||
|
||||
Update your Pleroma configuration like that in `/etc/pleroma/config.exs`.
|
||||
|
||||
```
|
||||
config :pleroma, Pleroma.Upload,
|
||||
uploader: Pleroma.Uploaders.S3,
|
||||
base_url: "https://pleroma.garage.example.tld"
|
||||
|
||||
config :ex_aws, :s3,
|
||||
access_key_id: "GW...",
|
||||
secret_access_key: "XXX",
|
||||
region: "garage",
|
||||
host: "api.garage.example.tld"
|
||||
```
|
||||
|
||||
And restart Pleroma.
|
||||
|
||||
You can found more information in [Pleroma Documentation > Pleroma.Uploaders.S3](https://docs-develop.pleroma.social/backend/configuration/cheatsheet/#pleromauploaderss3)
|
||||
|
||||
### Migrating your data
|
||||
|
||||
Pleroma have an internal migration tool that can encounter some fatal error
|
||||
|
||||
```
|
||||
** (EXIT from #PID<0.98.0>) an exception was raised:
|
||||
** (File.Error) could not stream "/var/lib/pleroma/uploads/09/f8": illegal operation on a directory
|
||||
(elixir 1.17.3) lib/file/stream.ex:100: anonymous fn/3 in Enumerable.File.Stream.reduce/3
|
||||
(elixir 1.17.3) lib/stream.ex:1675: anonymous fn/5 in Stream.resource/3
|
||||
(elixir 1.17.3) lib/stream.ex:1891: Enumerable.Stream.do_each/4
|
||||
(elixir 1.17.3) lib/task/supervised.ex:370: Task.Supervised.stream_reduce/7
|
||||
(elixir 1.17.3) lib/enum.ex:4423: Enum.map/2
|
||||
(ex_aws_s3 2.5.8) lib/ex_aws/s3/upload.ex:141: ExAws.Operation.ExAws.S3.Upload.perform/2
|
||||
(pleroma 2.10.0) lib/pleroma/uploaders/s3.ex:60: Pleroma.Uploaders.S3.put_file/1
|
||||
(pleroma 2.10.0) lib/pleroma/uploaders/uploader.ex:49: Pleroma.Uploaders.Uploader.put_file/2
|
||||
```
|
||||
|
||||
So, use [your best tool](https://garagehq.deuxfleurs.fr/documentation/connect/cli/) to sync `/var/lib/pleroma/uploads/` in your S3.
|
||||
|
||||
Then, to avoid some non existent problem (just in case of), run this command
|
||||
|
||||
```bash
|
||||
while true
|
||||
do
|
||||
rm -vr $(./bin/pleroma_ctl uploads migrate_local S3 2>&1 | grep "could not stream" | awk -F '"' '{print $2}')
|
||||
sleep 5
|
||||
done
|
||||
```
|
||||
|
||||
If you have many files, stop this command sometime and the command bellow (interactive) to delete local
|
||||
file after upload. Then restart the loop.
|
||||
|
||||
```bash
|
||||
./bin/pleroma_ctl uploads migrate_local S3 --delete
|
||||
```
|
||||
|
||||
And *voilà*
|
||||
[Pleroma Documentation > Pleroma.Uploaders.S3](https://docs-develop.pleroma.social/backend/configuration/cheatsheet/#pleromauploaderss3)
|
||||
|
||||
## Lemmy
|
||||
|
||||
|
|
@ -759,7 +585,7 @@ This feature requires `pict-rs >= 4.0.0`.
|
|||
This is the usual Garage setup:
|
||||
|
||||
```bash
|
||||
garage key create pictrs-key
|
||||
garage key new --name pictrs-key
|
||||
garage bucket create pictrs-data
|
||||
garage bucket allow pictrs-data --read --write --key pictrs-key
|
||||
```
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ Note that `git-annex` requires to be compiled with Haskell package version
|
|||
`aws-0.24` to work with Garage.
|
||||
|
||||
```bash
|
||||
garage key create my-key
|
||||
garage key new --name my-key
|
||||
garage bucket create my-git-annex
|
||||
garage bucket allow my-git-annex --read --write --key my-key
|
||||
```
|
||||
|
|
@ -207,13 +207,3 @@ $ plakar at @garageS3 ls
|
|||
```
|
||||
|
||||
More information in Plakar documentation: https://www.plakar.io/docs/main/quickstart/
|
||||
|
||||
## Synology HyperBackup
|
||||
|
||||
HyperBackup can be configured to upload backups to garage using a custom S3 destination. However, the HyperBackup client hardcodes the `us-east-1` region that is a critical input to the v4 signature process. If garage is not set to `us-east-1`, HyperBackup will recognize available buckets, but fail during the final setup stage.
|
||||
|
||||
In garage.toml:
|
||||
```toml
|
||||
[s3_api]
|
||||
s3_region = "us-east-1"
|
||||
```
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue